First date with Python

Posted on March 26th, 2010 by Daniel Nichter

I began learning Python a few months back by reading Programming in Python 3. I didn’t know at the time that Python 3 was devel and that Python 2 was production. Regardless, I got the chance yesternight to sit down and use Python in the real world. My first languages where BASIC (pre-Microsoft), Pascal and C–primarily the latter. Then I learned C++, PHP, JavaScript and finally Perl some years ago. Now Python.

I think Perl is a perfect language; I absolutely love it. Sure it’s not the fastest in all cases, and it’s memory consumption is epic at times, but Perl is beautiful, expressive and tremendously helpful. It’s been said that Python has a beautiful heart and I may agree because I’m attracted to simplicity. In Perl there’s always a multitude of ways to do things and no one way is “correct”, but I’ve read that in Python there’s one way to do things and that’s the Python way. I’ve also read that “Python is not C” (when I couldn’t figure out why i++ wouldn’t work). Python may be a little obstinate but as André Gide said, “Il vaut mieux d’être détesté pour ce qu’êtes vous que pour être aimé pour ce que n’êtes pas vous.”

It’s no use doing a strict side-by-side comparison because there’s very little similarity between the languages other than they’re higher-level than C. Since I use languages to express business ideas and solve business problems, my concern is efficacy; i.e. does this language help or hinder me? Of course, one’s skill in the language is an important factor, but that aside (because my skill in Python is not even half my skill in Perl), I feel Python itself was helpful. What I found to be a hindrance was the Python documentation.

The docu is fantastically detailed, but that’s also the problem. One of Perl docu’s greatest features is the SYNOPSIS section of modules. A language’s syntax and built-in/basic data types is the trivial part of its learning curve. To do something useful with the language requires modules/classes. At first I don’t care about the details of these things, I just want to know the most basic usage–I want a synopsis. From that I get a feel for the module/class and can intuit how the other stuff is going to work.

Perhaps that’s only a Perl influence due to Perl modules having zero standard interface. To Python’s advantage its classes seem to have a much more consistent look and feel. With more time I’ll probably learn to read pydocs more efficiently. Also, since there’s one correct Python way to do things, I may even achieve Python proficiency more quickly than I did Perl proficiency, which is an eternal struggle when it’s permissible to be lazy or obscure.

It’s odd but that’s what I look forward to discovering with Python: will it permit me to be lazy? C is an awesome language but it’s also a relentless task-master. Perl is both awesome and lazy. Perhaps Python will be somewhere in between.

Tags: Andé Gide, C, lazy, pydoc

Double negatives and double meanings

Posted on March 16th, 2010 by Daniel Nichter

In formal logic ¬¬ok is ok (pun intended) but in code !not_ok is confusing. This is also wildly confusing to me: cache = 1 unless defined cache. So caching is enabled if cache is not defined, which means both cache = 0 and cache = 1 disable caching. This is a case of double meaning which is the evil twin of double negative.

Here’s another gem of my own doing:

--[no]buffer-to-client

    default: yes

    Fetch rows one-by-one from MySQL while comparing.

    This is disabled by default.  If enabled, all rows will be fetched
    into memory for comparing.  This may result in the results "cursor"
    being held open for a shorter time on the server, but if the tables
    are large, it could take a long time anyway, and eat all your
    memory.  For most non-trivial data sizes, you want to leave this
    disabled.

So is it enabled or disabled by default? I should know but sadly I don’t.

Tags: double meaning, double negative

Perl SQL parser

Posted on March 6th, 2010 by Daniel Nichter

I know “great programmers reuse code”, or some adage like that, but I wrote my own SQL query parser in Perl for various, uninteresting reasons. My goals were:

1. Single package/file without external dependencies (rules out SQL::Parser)
2. MySQL-specific, i.e. parse MySQL’s “flexible” syntax
3. Full FROM/JOIN clause and subquery parsing
4. Parse nested clauses/statements/(sub)queries
5. Parse everything into logical structure (logical to me at least)
6. Extensively tested
7. Handle 90% of cases

The result is SQLParser.pm (only available via that link, not on CPAN).

It successfully parses:

select
   now(),
   (select foo from bar where id=1)
from
   t1,
   t2
   join (select * from sqt1) as t3 using (`select`)
   join t4 on t4.id=t3.id
where
   c1 > any(
      select col2 as z from sqt2 zz where sqtc<(
         select max(col) from l where col<100
      )
   )
   and s in ("select", "tricky")
   or s <> "select"
group by 1
limit 10

(That syntax is valid but the query is a complete fabrication, i.e. not sane.) Apart from the MySQL source I don’t know of any other publicly available code that can successfully parse a SQL statement like that. Granted, my code has limitations, too, like CASE statements, but my goal was only 90% of cases (relative; what’s 90% for me may not be 90% for you).

The code was easier than I thought at first. Parsing FROM/JOIN clauses and subqueries are the biggest challenges. There’s a fair amount of code comments so you should be able to follow along and understand why and how I do things.

If you know of a better Perl SQL parser that meets my goals, please let me know!

Tags: parsing, SQL