On token boundaries

If I asked you to separate helloworld into meaningful words, you would immediately jump to hello and world before your mind could even consider that the first word might be he or hell. You’ve just created a token boundary using only a pre-existing knowledge of the language. In other words, the token boundary wasn’t encoded in the source material — you used outside information to infer one. This isn’t how parsers work. ...

October 19, 2024 · Tanay Gupta

An extraordinary code editor - LibreOffice Calc

Spreadsheet software lacks syntax highlighting, autocomplete, code browsing, and perhaps any other feature you might want out of a code editor. However, there is one aspect of programming in which LibreOffice Calc excels beyond its classical contemporaries. In this thought experiment, allow me to show you what spreadsheets have to offer to the design of computer languages. A different medium Plain-text1 is an unbroken sequence of symbols extending to infinity. It is purely one-dimensional. We are used to seeing rows and columns in our text files, but this is merely the perception our tools grant us for our convenience. The symbols are still arranged one after the other, in a single dimension. ...

September 23, 2024 · Tanay Gupta

Collisions in computer languages

Hey there! I’m glad you are here. I was looking around in my old stuff and I just found my old hard-drive. I bought some Bitcoin back when it was worth nothing, and this is the drive that contains my old Bitcoin wallet! Let’s take a look together. $ tree . . └── bitcoin └── wallet.txt It’s all coming back to me now. I bought 10 Bitcoin for a dollar each years ago. I lost my drive after we moved, and after all that time I’ve found it again! ...

August 17, 2024 · Tanay Gupta