February 1, 2009

Why use Spring Python and not just plain python?

Filed under: python — gregturn @ 8:17 am

I have bumped into articles and forum postings, blog entries, other blog entries, and yet more blog entries. In fact, while writing this blog entry, someone posed the question “what is the aim of Spring Python?” (Click and see my answer.) There are a diverse range of view points, and many in disagreement about the viability of DI in python.

Many of these opinions raise a good question. Why do you need some complicated thing like Spring Python when you could be using pure python to do your job? I was very impressed with Jamis’ article LEGOs, Play-Doh, and Programming. The core question is, why use complicated Java idioms that can be easily and dynamically expressed in pure ruby? I value this article because it encourages me to recheck my assumptions and ideas. It doesn’t matter how good your solution is. If you built it on a platform of invalid, unwise, or ridiculous assumptions, you probably aren’t going anywhere fast.

So, what ARE my assumptions?

  • Being able to apply extra services to my code without touching the code is good.
  • Being able to inject test dummies into my code without editing my code is good.
  • Simplifying and reducing code volume is good.
  • Using fancy tricks that are hard to read the next day is bad.
  • Using Problem A’s solution for Problem B without analysis is dumb.

I want to compare this against what some people may think my assumptions are.

  • XML is the way to configure everything.
  • Java’s solutions are perfect for Python.
  • Dependency Injection should be used for every problem.

Let’s look at this in closer detail. Hopefully, I can show some insight into places where I think Spring Python can help, and other places it would be overkill to use it. Nothing works everywhere, right?

Being able to apply extra services to my code without touching the code is good – True

The first couple of things I can think of are transactions and security. These are functions, that when you implement them by hand, are a pain in the neck! You keep writing the same boilerplate plumbing code wrapping logical blocks, and you have to do it right every time, or you may find yourself in a big hole.

Sure, any part of your code not working correctly is an issue. But this isn’t YOUR code we are talking about. This is simply a layer of protection you are trying to wrap around code to make it safe. There isn’t a whole lot of logic in transactions or security, just simple, copy-and-pasted stuff you need. And we know what risk we are at when copy-and-paste starts happening.

That is the definite code smell that AOP is needed. Spring Python provides an AOP solution. You can write any type of advice you need. But considering how prevalant transactions and security are, we went ahead and coded a flexible, pythonic way to do it.

But don’t forget. The premise is that this is applied without modifying your working code or your calling code. If only we could have something that sat between the business code and the “main” function that calls the code, which managed instantiation of dependencies, and which we could modify with no impact…..you mean, like an IoC container?

I know what you’re thinking. “Oh no! Not that XML stuff! I want to stick to python.” Who said you have to use XML? You can code your transactions simply using python decorators and a pure python container. XML is only provided to help people coming from Java-land migrate to Python. Some people like XML. Some people don’t. Its a choice, not a requirement.

I know there is something else to consider. If I’m using TDD and I am spotting the places where I need to make changes, why can’t I just make the changes right then and there? If there is only one time and one place where you are injecting your AOP advice, you are probably right. It is when you need to apply one piece of advice in multiple places where you start doing the copy-and-paste routine. And when you need to add more advice, and repeat this whole process over again where things get costly. Instead, one refactoring to expose that service as an injection point, and you can easily adjust things in the future.

Being able to inject test dummies into my code without editing my code is good – True

Testability of dynamic code is key. We preach that dynamic typing works, provided you adequately test your code. If you remember anything from your computer engineering courses in college, the more components you have, the greater the number of permutations of state. Being able to isolate functionality is the way to increase confidence in test cases, and that means we need to be able to replace production dependencies with test doubles.

It is easy to get caught up in the “I want to inject everything” hype. Don’t! Instead, only inject key things. TDD will expose what test doubles you want to inject, and thus what parts need to be wired by IoC. If your application is small enough, you may not need any at all. And that’s alright! Perhaps you start off, and all you need to inject is a database test double. Assuming you have defined MyApp and DataAccess, your container definition can be as simple as:

class MyAppContext(PythonConfig):
    def __init__(self):
        super(MyAppContext, self).__init__()
 
    @Object
    def my_app(self):
        return MyApp(self.data_access())
 
    @Object
    def data_access(self):
        return MyDataAccess()

For your test case, you just need an alternative:

class MyTestableAppContext(MyAppContext):
    def __init__(self):
        super(MyTestableAppContext, self).__init__()
 
    @Object
    def data_access(self):
        return DataAccessStub()

How hard was that? You just extend your existing container, overriding a key injection point, and poof! You’re there with a nicely isolated, testable version of your code! I don’t know about you, but that is arguably a DSL, written in pure python.

Simplifying and reducing code volume is good – True

I mentioned transactions earlier. Let’s look at an ugly example of some code with transactions:

class Bank:
    def __init__(self):
        self.factory = factory.MySQLConnectionFactory("springpython", "springpython", "localhost", "springpython")
        self.dt = DatabaseTemplate(self.factory)
 
    def balance(self, account_num):
        results = self.dt.query_for_list("select BALANCE from ACCOUNT where ACCOUNT_NUM = %s", (account_num,))
        if len(results) != 1:
            raise InvalidBankAccount("There were %s accounts that matched %s." % (len(results), account_num))
        return results[0][0]
 
    def withdraw(self, amount, source_account_num):
        if self.balance(source_account_num) < amount:
            raise InsufficientFunds("Account %s did not have enough funds to transfer %s" % (source_account_num, amount))
        self.dt.execute("update ACCOUNT set BALANCE = BALANCE - %s where ACCOUNT_NUM = %s", (amount, source_account_num))
 
    def deposit(self, amount, target_account_num):
        self.balance(target_account_num) # Implicitly testing for valid account number
        self.dt.execute("update ACCOUNT set BALANCE = BALANCE + %s where ACCOUNT_NUM = %s", (amount, target_account_num))
 
    def transfer(self, transfer_amount, source_account_num, target_account_num):
        try:
            cursor = self.factory.getConnection().cursor() # DB-2.0 API spec says that creating a cursor implicitly starts a transaction
            self.withdraw(transfer_amount, source_account_num)
            self.deposit(transfer_amount, target_account_num)
            self.factory.getConnection().commit()
            cursor.close() # There wasn't anything in this cursor, but it is good to close an opened cursor
        except InvalidBankAccount, InsufficientFunds:
            self.factory.getConnection().rollback()

If you can rip out that plumbing code, your code becomes cleaner, easier to read, and easier to maintain. The following shows the code where you don’t have to explicitly manage the transaction. Instead, it is declared using a python decorator.

class Bank:
    def __init__(self):
        self.factory = factory.MySQLConnectionFactory("springpython", "springpython", "localhost", "springpython")
        self.dt = DatabaseTemplate(self.factory)
 
    def balance(self, account_num):
        results = self.dt.query_for_list("select BALANCE from ACCOUNT where ACCOUNT_NUM = %s", (account_num,))
        if len(results) != 1:
            raise InvalidBankAccount("There were %s accounts that matched %s." % (len(results), account_num))
        return results[0][0]
 
    def withdraw(self, amount, source_account_num):
        if self.balance(source_account_num) < amount:
            raise InsufficientFunds("Account %s did not have enough funds to transfer %s" % (source_account_num, amount))
        self.dt.execute("update ACCOUNT set BALANCE = BALANCE - %s where ACCOUNT_NUM = %s", (amount, source_account_num))
 
    def deposit(self, amount, target_account_num):
        self.balance(target_account_num) # Implicitly testing for valid account number
        self.dt.execute("update ACCOUNT set BALANCE = BALANCE + %s where ACCOUNT_NUM = %s", (amount, target_account_num))
 
    @transactional(["PROPAGATION_REQUIRED"])
    def transfer(self, transfer_amount, source_account_num, target_account_num):
        self.withdraw(transfer_amount, source_account_num)
        self.deposit(transfer_amount, target_account_num)

If you focus your attention on the transfer method, you should notice an improvement in readability. At first, maybe it doesn’t like much. But try to visualize cleaning up a whole slew of transactional blocks of code you have to maintain, and the benefits starts to add up.

Also, I don’t know if you noticed, but the withdraw and deposit operations aren’t safe enough to be part of the public API. They need to be transactional as well. So why aren’t they? Well, I wanted this to be an apples-to-apples comparison. You can’t just blindly wrap the first sample of deposit with a transaction, because the transfer function may have already started a transaction. You need to test to see if you are already part of a transactions.

Guess what: @transactional handles this for you. The second example is easy. Just put @transactional in front of withdraw and deposit and you are done. The PROPAGATION_REQUIRED tag indicates a transaction will be started if one doesn’t already exist. Doing this with the first example would have required hand-written code to track whether you were in or out of transactions.

class Bank:
    def __init__(self):
        self.factory = factory.MySQLConnectionFactory("springpython", "springpython", "localhost", "springpython")
        self.dt = DatabaseTemplate(self.factory)
 
    def balance(self, account_num):
        results = self.dt.query_for_list("select BALANCE from ACCOUNT where ACCOUNT_NUM = %s", (account_num,))
        if len(results) != 1:
            raise InvalidBankAccount("There were %s accounts that matched %s." % (len(results), account_num))
        return results[0][0]
 
    @transactional(["PROPAGATION_REQUIRED"])
    def withdraw(self, amount, source_account_num):
        if self.balance(source_account_num) < amount:
            raise InsufficientFunds("Account %s did not have enough funds to transfer %s" % (source_account_num, amount))
        self.dt.execute("update ACCOUNT set BALANCE = BALANCE - %s where ACCOUNT_NUM = %s", (amount, source_account_num))
 
    @transactional(["PROPAGATION_REQUIRED"])
    def deposit(self, amount, target_account_num):
        self.balance(target_account_num) # Implicitly testing for valid account number
        self.dt.execute("update ACCOUNT set BALANCE = BALANCE + %s where ACCOUNT_NUM = %s", (amount, target_account_num))
 
    @transactional(["PROPAGATION_REQUIRED"])
    def transfer(self, transfer_amount, source_account_num, target_account_num):
        self.withdraw(transfer_amount, source_account_num)
        self.deposit(transfer_amount, target_account_num)

To tell you the truth, writing functional code I can maintain is my #1 goal in software development. I have inherited many projects that were impossible to read. They worked, but because they were so poorly done, I couldn’t enhance or improve them, and thus began the refactor/rewrite.

Using fancy tricks that are hard to read the next day is bad – True

Only use Spring Python if you need it. And only use what you need. If you really need to mark up transactions in your code, this is perfect. But if your tool doesn’t need that protection, forget about it! If your tool needs rigorous testing, utilize the IoC container to inject test doubles. But if you are writing a simple admin tool, then perhaps you don’t need any IoC at all. Same goes for all the features.

If you need IoC and you are used to reading XML-configured app contexts (perhaps because you used to write Java code a lot), then you can easily utilize the functionality of Spring Python. But if you have been a pythonista since day one, forget about it! Use the pure python, decorator based container, and don’t waste one minute on XML.

Using Problem A’s solution for Problem B without analysis is dumb – True

Just because an AOP transactional interceptor was what you needed for Problem A, doesn’t mean you need it for Problem B. And don’t assume that your strategy of IoC injection points for Problem A will map cleanly to Problem B. Again, TDD should help you expose the injection points. I can’t possibly predict where you need to inject dependencies. That is what software analysis is for.

Anyone can pick up and use a library. But using it for your needs (and not mine) is where judgment calls come in. Your experience in using tools to solve problems is needed to figure out what parts of Spring Python will serve your needs.

XML is the way to configure everything – False

This is a false assumption, and unfortunately an oft repeated one. Notice that there was no XML to be found in my samples? I stuck with pure python, using what I argue is similar to a DSL. Consider writing your code to have some fluent APIs, and the pure python IoC container can easily and efficiently wire your application together as needed. I have also done my best to update documentation to show configuring things either way. Whether or not the documentation is up-to-date, please remember, you can ALWAYS stick with pure python.

Java’s solutions are perfect for Python – False

I already wrote how Spring Python is not just a simple port. The idea is to grab higher level concepts that transcend languages, and see where they apply. Some perceive this as grabbing Java’s solutions and dropping them into Python. It’s not. I know a dozen or so computer languages (C, C++, Tcl, tk, expect, PERL, java, python, csh, bash, Forte 4GL, and more), and some problems I find occur no matter where I go. Some carry a similar solution, some don’t. The key factor is, many of these problems need a solution that looks similar from 10,000 feet, but when you look deeper, the details are very different. That is what Spring Python strives for: solving some of the same high level concepts, where the high level perspective looks similar, but the details use the power and idioms of python. If you perceive one of my ideas as being out-of-whack, please drop me a comment, and I will be happy to discuss it.

Dependency Injection should be used for every problem – False

You need look no further than at coily, our command line tool, to see that some solutions don’t require any Spring Python. Instead, good ole’ raw python does the trick. Use Spring Python sparingly, and when it improves your efficiency, NOT just for the sake of using. This is the type of critical judgment you should apply to any library you utilize. Use what you need to solve your TDD test cases, and when they pass STOP! Move away from the keyboard. You are done.

Conclusion

I will say I have presented at least a minimal case that I’m not really using Java idioms, but instead solving higher level problems. People may assume I am reusing the idioms considering my library was inspired by one written in Java. But you really need to peel back the layers and see what is happening. This is true for any library you use. Look at the documentation we provide and inspect the source code, and see if you can spot any anonymous inner classes, or getter & setters. You won’t find them. Instead, you find anonymous functions, python iterators, usage of standard python libraries, etc, and what I consider a slim-and-trim library of functionality. And on top of that, lots of documentation on how to use it along with a solid forum of support. Makes it more than just a random recipe posted somewhere, and subject to discovery by google.

VN:F [1.9.22_1171]
Rating: 5.0/5 (5 votes cast)
VN:F [1.9.22_1171]
Rating: +2 (from 2 votes)
Why use Spring Python and not just plain python?, 5.0 out of 5 based on 5 ratings
Be Sociable, Share!

Comments are closed.