Unexpected Indent Fix For Python Org Mode Scripts

THE PROBLEM

I use Emacs Org Mode to manage my notes. It's a plain-text system who's killer feature is the ability to run code and output the results inside the notes themselves. It's like Jupyter Notebooks but available directly in the editor with the ability to run any language.

It's awesome.

But, I kept running into issues with Python snippets. Auto formatting sometimes adds spaces to otherwise empty lines. Python's white-space sensitive interpreter can't figure out what to do with them so it pukes.

For example:

def ping():
    print("alpha")
    
if __name__ == "__main__":
    ping()
  File "<stdin>", line 4
    if __name__ == "__main__":
    ^^
SyntaxError: invalid syntax
  File "<stdin>", line 1
    ping()
IndentationError: unexpected indent

The code looks fine, but the formatter added spaces to line 3 that broke it.

THE SOLUTION

The issue doesn't occur all the time. It's not hard to clean up manually, but it happened frequently enough that I went after a fix. This is what I came up with:


(add-hook
 'org-ctrl-c-ctrl-c-hook
 (function
  (lambda ()
    (replace-regexp-in-region
     "\s+\n"
     "\n"
     (org-element-property :begin
                           (org-element-at-point))
     (org-element-property :end
                           (org-element-at-point)))
    nil)))

That code creates a hook the runs before code blocks get executed. It works by running a regular expression find and replace over the source block looking for lines that contain only spaces and flattening them down so they just contain a newline character. The Python interpreter has no problem with those.

With that in place, the exact some code works as expected.

def ping():
    print("alpha")

if __name__ == "__main__":
    ping()

alpha

THE SHIFTS

An important point is that if you have multiple things that should be in the same block, they either need to be directly connected or have some type of filler line connect them. Classes are an example of this. The following will break because there is space between the two def statements.

class Example():
    def ping(self):
        pass

    def pong(self):
        pass

  File "<stdin>", line 1
    def pong(self):
IndentationError: unexpected indent
  File "<stdin>", line 1
    pass
IndentationError: unexpected indent

The fix is to connect things directly like this:


class Example():
    def ping(self):
        pass
    def pong(self):
        pass

Or via a comment like this:


class Example():
    def ping(self):
        pass
        #
    def pong(self):
        pass

That's not ideal, but at least it's visible. The comment can also go anywhere on the line.

I may play around with making a version that doesn't require sometime of connection like that. I expect that would be way more complicated though. This is working fine for me for now.

USAGE

I maintain both my website and configurations for emacs directly inside my notes. So, the code you see above is the real thing. Here's a link to my process for setting it up. While it's a bit of hassle to setup it works great. Of course, you don't have to go that route. Any way that gets the code to run will work.

I've really enjoyed most of my experience with emacs, but those python errors were stacking up against it. I'm really glad to have that solved.


Notes

  • One effect of this approach is that you can't have empty lines between code that should be connected. I don't find that a problem in general, but if I want a little visual separation I use a # to comment the line so it doesn't get modified.

  • It took eight straight hours to figure out those twelve lines of code. I posted my notes on that process if you're interested in the iterations.

  • The hook alters the original source block to remove the spaces. That doesn't bother me. If it become a problem adding code to avoid that is totally possible.

  • The hook currently runs on source blocks for all languages. That hasn't been a problem. If it becomes one, I'll update it to just adjust python source blocks.