Moved. See All new content goes to the new site. This is a legacy, and will likely be dropped five years after the last post in Jan 2023.

Tuesday, August 25, 2015

Visual studio and Python

Why write Python in Visual Studio?

That what I want to know, too.

IntelliSense? ActiveState Komodo does this. And it does it very well considering the potential complexity of trying to determine what identifiers are possibly valid in a dynamic language.

Debugger? No thanks. I haven't used it yet. [I should probably blog on the perils of debuggers.]

Project Management? GitHub seems to be it. Some IDE integration might be helpful, but the three common command-line operations -- git pull, git commit, and git push -- seem to cover an awful lot of bases.

I've been asked about Python IDEs -- more than once -- and my answer remains the same:
The IDE Doesn't Matter. 

One of the more shocking tech decisions I've seen is the development manager who bragged on the benefits of VB. The entire benefit was this: Visual Studio made the otherwise awful VB language acceptable.

The Visual Studio IDE was great. And it made up for the awful language.


The development manager went to to claim that until Eclipse had all the features of Visual Studio, they were sure that Java was not usable. To them, the IDE was the only decision criteria. As though code somehow doesn't have a long tail of support, analysis, and reverse engineering. 

Tuesday, August 18, 2015

Audio Synth [Updated]

I learned about synthesizers in the '70's using a Moog analog device. Epic coolness.

Nowadays, everything is digital. We use wave tables and (relatively) simple additive synth techniques.

I made the mistake of reading about Arduino wave table synthesis:

The idea of an Arduino alarm that uses a chime instead of a harsh buzz is exciting. The tough part about this is building the wave tables.

What a perfect place to use Python: we can build wave tables that can be pushed down to the Arduino. And test them in the Python world to adjust the frequency spectrum and the complex envelope issues around the various partials.



Python3.4 doesn't have PyAudio support.


Sigh. Before I can work with Arduino wave tables, I'll have to start by figuring out how to build PyAudio for Python 3.4 on Mac OS X.

Look here: for the code.

Look here for the secret to building this on Mac OS X:


  1. Get pyaudio source.
  2. Inside pyaudio create a portaudio-v19. Get the portaudio source and put it here.
  3. Inside pyaudio/pyaudio, do ./config; make and sudo make install
  4. Inside pyaudio, do python3.4 install --static-link

Tuesday, August 4, 2015

Mocking and Unit Testing and Test-Driven Development

Mocking is essential to unit testing.


It's also annoyingly difficult to get right.

If we aren't 100% perfectly clear on what we're mocking, we will merely canonize any dumb assumptions into mock objects that don't really work. They work in the sense that they don't crash, but they don't properly test the application objects since they repeat some (bad) assumptions.

When there are doubts, it seems like we have to proceed cautiously. And act like we're breaking some of the test-first test-driven-development rules.

Note. We're not really breaking the rules. Some folks, however, will argue that test-driven development means literally every action you take should be driven by tests. Does this include morning coffee or rotating your monitor into portrait mode? Clearly not. What about technical spikes?

Our position is this.
  1. Set a spike early and often. 
  2. Once you have reason to believe that this crazy thing might work, you can formalize the spike with tests. And mock objects.
  3. Now you can write the rest of the app by creating tests and fitting code around those tests.
The import part here is not to create mocks until you really understand what you're doing.

Book Examples

Now comes the tricky part: Writing a book.

Clearly every example must have a unit test of some kind. I use doctest heavily for this. Each example is in a doctest test string.

The code for a chapter might look like this.

test_hello_world = '''
>>> print( 'hello world')
'hello world'

__test__ = { n:v for n,v in vars().items() 
    if n.startswith('test_') }

if __name__ == '__main__':
    import doctest

We've used the doctest feature that looks for a dictionary assigned to a variable named __test__. The values from this dictionary are tests that get run as if they were docstrings found inside modules, functions, or classes.

This is delightfully simple. Expostulate. Exemplify. Copy and Paste the example into a script for test purposes and Exhibit in the text.

Until we get to external services. And RESTful API requests, and the like. These are right awkward to mock. Mostly because a mocked unittest is singularly uninformative.

Let's say we're writing about making a RESTful API request to The results of the request are very interesting. The mechanics of making the request are an important example of how REST API's work. And how CKAN-powered web sites work in general.

But if we replace urrlib.request with a mock urllib, the unit test amounts to a check that we called urlopen() with the proper parameters. Important for a lot of practical software development, but also uniformative for folks who download the code associated with the book.

It appears that I have four options:

  1. Grin and bear it. Not all examples have to be wonderfully detailed.
  2. Stick with the spike version. Don't mock things. The results may vary and some of the tests might fail on the editor's desktop.
  3. Skip the test.
  4. Write multiple versions of the test: a "with real internet" version and a "with corporate firewall proxy blockers in place" version that uses mocks and works everywhere.
So far, I've leveraged the first three heavily. The fourth is awkward. We wind up with code like this:

class Test_get_whois(unittest.TestCase):
    def test_should_get_subprocess(self):
        subprocess = MagicMock()
        with patch.dict('sys.modules', subprocess=subprocess):
            import subprocess
            from ch_2_ex_4 import get_whois
            result= get_whois('')
        self.assertEquals( result, ['', 'words'] )
        subprocess.check_output.assert_called_with(['whois', ''])

This is not a lot of code for enterprise software development purposes. It's a bit weak, in fact, since it only tests the Happy Path.

But for a book example, it seems to be heavy on the mock module and light on the subject of interest.
Indeed, I defy anyone to figure out what the expository value of this is, since it has only 2 lines of relevant code wrapped in 8 lines of boilerplate required to mock a module successfully.

I'm not unhappy with the unitest.mock module in any way. It's great for mocking modules; I think the boilerplate is acceptable considering what kind of a wrenching change we're making to the runtime environment for the unit under test.

This fails at explication.

I'm waffling over how to handle some of these more complex test cases. In the past, I've skipped cases, and used the doctest Ellipsis feature to work through variant outputs. I think I'll continue to do that, since the mocking code seems to be less helpful for the readers, and too focused on purely technical need of proving that all the code is perfectly correct.