Jan 04

An advanced cryptanalysis problem

Can you break the code of the JC Penney gift receipt? One digit isn't represented below - can you deduce its encoded value anyway? Select the contents of the box below (drag over it) for the answer.

1=Q, 2=W, 3=E (probably - look at your keyboard), 4=R, 5=T, 6=Y, 7=U, 8=I, 9=O, 0=P


Posted by Erik | Permanent link

Dec 20

Do users matter?

The title is facetious, but it got your attention, no?

My day job is in a hard-core Windows shop. How hard-core? A visiting exec recently told us in a meeting that once chaired a committee whose purpose was to "advance the spread of Microsoft technologies throughout the enterprise".

Yesterday the consequences of this culture were set in sharp relief by this conversation. For context, SSIS is a technology used mostly for loading flat files into databases, and it's meant to be a replacement for the older DTS platform.

Erik: Hey, Lydia, can you help me with this SSIS error?

Lydia: Maybe. I've played with SSIS, but usually I'm in a hurry so I bust out a DTS package instead.

Erik: I feel better to hear you say that. Everytime I change an SSIS package, it takes hours and hours. It's so over-engineered. I think Shane originally pushed for us to use this instead of, say, a BULK INSERT.

Lydia: Oh no, Shane wouldn't have done that. The last time I mentioned using SSIS, the whole team (and Shane) told me not to go there [stop-sign hand gesture], that we don't need the complexity.

Erik: I see. Well, if I can't get around this error, could I convert this to DTS or a BULK INSERT?

Lydia: If we have to, we can. But really, we should be using SSIS. Eventually, DTS will be retired, so we should use SSIS instead. Use SSIS if at all possible...

See how that works? We have a proven technology that we like and can work in very quickly. But the vendor is taking it away, so we really should use the over-engineered, non-productive solution, since eventually we'll have no choice.

As the token Linux guy in the office, I get pigeon-holed as a Microsoft-hater. I'm not, really, but that's the stereotype. Exchanges like this, though, hardly make me want to "advance the spread of Microsoft technologies".


Posted by Erik | Permanent link

Dec 14

Comprehensive source backup

I've wanted a way to back up all my projects to offsite storage in the laziest way possible. Here's the function in, you guessed it, my .bashrc:

 1 # Back up all repos off site
 2 function bak {
 3         # mount my project partition
 4         m1
 5
 6         # Find and iterate all git repos
 7         sudo find /* -type d -name ".git" |
 8         while read repo; do
 9
10                 # Ignore these repos
11                 [ $repo == "/home/erik/tests/ffExtOne/.git" ] && continue
12                 [ $repo == "/home/erik/tests/ffExtTwo/ext/.git" ] && continue
13                 [ $repo == "/home/erik/tests/glibOne/.git" ] && continue
14                 [ $repo == "/home/erik/djangosites/yystclair/.git" ] && continue
15                 [ $repo == "/mnt/hda1/devel/projects/gnc/repo/.git" ] && continue
16                 [ $repo == "/mnt/hda1/devel/gitSandbox/.git" ] && continue
17                 [ $repo == "/mnt/hda1/devel/gitSandbox/ignoreTest/.git" ] && continue
18
19                 # Send repo to remote server
20                 sudo rsync --verbose --stats --compress --relative --progress --recursive --times --delete $repo rsync://xxx.net/MyRsyncHomeForSource
21         done
22 }

It works by scanning the entire filesystem for .git directories, and rsync'ing them to my offsite location. By using the --relative flag, the full paths are copied under a single remote rsync location, instead of just .git (which would collide with prior .git's). The full path is enough to remind me what's in each repository.

When I create a new repo for a project, I don't have to take a single extra step to make sure it's part of the source backup. I can optionally specify repos to ignore, but the function errs on the side of inclusion.

It took about 20 minutes to run the first time. Since subsequent backups will be diffs, it will be faster in the future. Using locate instead of find could speed up the process, as long as I was sure to updatedb prior to pushing to make sure that any brand-new repos were included.


Posted by Erik | Permanent link

Nov 23

The mysteries of GLib

The more I explore GLib, the more difficulty I have with its design choices. I'll certainly find GLib useful, and I'm not dismissing it, but some promises have not been fulfilled.

The not-so-quiet main loop

GLib provides an interesting main loop feature, which allows your programs (console, daemon, not just UI) to be event-driven instead of procedural. What seems limiting is that only file descriptors can be event sources. Everyone knows that in Unix, everything is a file, right? So, why would you want an event source that's not a file descriptor? Oh, maybe because I'd like to declare events in the "user space" of my program, and not solely on low-level IO events.

The rationale given is that the Unix poll() function is the heart of the main loop. By using poll, the call is supposed to block until an interesting event occurs, sparing your CPU the incessant fast loop cycles. The only problem is, poll() doesn't work that way. Even the Unix man page seems confused about what the POLLIN flag means.

POLLIN There is data to read.

The behavior of poll() with POLLIN, however, is that it returns immediately with a POLLIN revent even when there is no data to read. That's because POLLIN really means "calling read() now won't block because the device is ready". POLLIN does not mean that there is data to read. This means your GLib app's check() must also call read() on every iteration to discern if there is actually data to read. It also means the CPU-saving long-blocking-until-an-event is actually a tight, wasteful quick-spinning loop. Maybe I should just put blocking read()s into dedicated threads.

Abstracting the wrong thing

So, since GLib is cross-platform, it abstracts away the platform-specific code, right? Not so with sockets. While a GIOChannel provides wrappers to do things like read a line from a file descriptor, there is no method given to create the file descriptor in the first place. So, to use sockets, I still need the #ifdef's for Windows-vs-Unix that I would write if I wasn't using a portability library.

The funny thing is, reading a line from a socket file descriptor (using recv()) is already the same on Windows/Unix. The priority seems to have been on making common interfaces for all file descriptors (files/sockets/pipes) instead of making common interfaces for the OS's in question (Windows/Unix).

One last thing...

GTK's combo boxes are oogly. Shoot, they're weird on Linux. There's no effort to use a native widget in Windows, and I'm sure on a Mac it looks even less native. Am I missing something here? Is there something special about MFC's licensing that forbids creating a GTK wrapper and requires a total re-write? It seems petty, and it's not a GLib-specific issue, but it could be a enough to make me look at a different library for cross-platform UI development.

In conclusion

Again, GLib appears great and I'm sure I'll use it. You get regex, a lexical scanner, and plenty of collection types. I just hope that when I explore those corners I won't encounter so many gotchas.


Posted by Erik | Permanent link

Nov 16

IRC done right with ii

IRC is a guilty pleasure. Logically, I know I've never gotten any value out of it. It's largely vacuous chat. Hanging out on ##c on freenode, any reasonable lurkers like myself are mesmerized by the comedy of people who are a) students trying to get homework "help" or b) embittered veterans who enjoy helping people like Nurse Ratchet enjoys helping her patients.

Nonetheless, I can't stay away. I guess I like the activity clicking down my screen every few seconds - I feel important and connected (don't think about that too hard).

ii is the only IRC client that has made sense to me. Rather than try to wrap the protocol into a custom interface with a custom scripting setup, ii gives you the protocol as a filesystem, and lets you define your own interface & pluggability through standard tools or shell capabilities. And at < 500 lines of dependency-free code, it's light as a feather.

So, how to use ii effectively? You guessed it, I have a shell function! The function sends the output of one channel to the console, formatted my way (= no presence messages or timestamps). screen is used to set up multiple windows.

Sending a message requires echo'ing to a flat file, but since I'm a lurker it's fine with me if it's hard to accidentally spam a channel when I think a different window is focused (a sadly common thing in a tiling window manager).

Here's the shell function from my .bashrc

# Usage: jn \#\#c  -- or -- jn \#\#c irc.freenode.net
# First form defaults to irc.freenode.net

function jn {

        IRCNAME=aurous

        # init
        IRC_HOST=irc.freenode.net
        if [ $2 ]; then
                IRC_HOST=$2
        fi

        SERVER_ROOT=~/irc/$IRC_HOST
        CHANNEL_ROOT=$SERVER_ROOT/$1

        # Truncate old irc log to last few lines
        if [ -f $CHANNEL_ROOT/out ]; then
                tail -n 1000 $CHANNEL_ROOT/out > $CHANNEL_ROOT/outnew
                rm $CHANNEL_ROOT/out
                mv $CHANNEL_ROOT/outnew $CHANNEL_ROOT/out
        fi

        # Start ii for this server if not running
        #if   ! ( ii procs   get command-line                      started with same server
        if( [ ! "`pgrep ii | xargs -I xxx echo /proc/xxx/cmdline | xargs -I xxx grep $IRC_HOST xxx`" ] ); then
                ii -n $IRCNAME -s $IRC_HOST &
                sleep 1
        fi

        # Join the channel
        echo "/j $1" > $SERVER_ROOT/in
        sleep 1

        # Check that we're connected to the channel
        if [ -f $CHANNEL_ROOT/out ]; then
                # tail
                tail -f $CHANNEL_ROOT/out | 
                while read ln; do 
                        #        strip presence lns   strip date/time stamp           let grep hilite nicks
                        echo $ln | grep -v "\-\!\-" | sed -e 's/^.................//' | grep "<\(.*\)>"
                done
        else
                echo Failure, couldnt read $CHANNEL_ROOT/out, probably no such channel
        fi
}


Posted by Erik | Permanent link

Nov 09

Handing over the keys

A couple of years ago I was nuts about DotNetNuke. I followed the releases, learned all the little corners of it, and even wrote a to-do app for sale. Now, everything's changed. I don't even have Windows readily accessible, let alone the development tools that make hacking on DotNetNuke fun. This is fine with me - after tasting Django, I know now how elegant web framworks can be.

I've been committed to DotNetNuke, though, since I did a DNN brochure site for a friend who makes and sells high-end purses. I've hung onto some crappy shared DNN hosting, and secretly hoped that Meg wouldn't ask for anything too complicated. Raw deal for Meg and me both.

So, my weekend project was to move her site from DNN into Django. This has several advantages:

  • No database (with Django, you add the DB when you need it)
  • Much simpler maintenance.
  • An interesting technology
  • Sensible URL's
  • A wget will pull a working static site

Wget it?

Let me elaborate on that last point. Meg and I are considering "handing the keys" of the site over to another guy, for reasons entirely unrelated to technology. How do you do that?

It's certainly not right to just give the new developer, a stranger, your DNN credentials and say "Have fun" (and anyway, this wouldn't relieve me of my DNN hosting). It's slightly less cruel to do this with a Django site. Alternatively, I may convert the Django extract into PHP, the lowest common denominator. I'm starting to think that the humane thing to do is to zip up a static web site, and let them set up the templating in their favorite technology (and hosted elsewhere).

Which brings me to a flaw in DotNetNuke: You can't do this. You can't wget a site and expect to navigate through the returned content and have the same experience as navigating in the original site. DNN uses redirects and has a complicated URL scheme.

Django, which makes URL design a first-class business process, makes your site easy to wget.

The lesson

There's a moral in here, and I'm still trying to put my finger on it. Candidates:

  • Don't use convoluted web frameworks with required databases and crusty, redirecting URL schemes.
  • Use only the most popular technologies.
  • Design for transfer - use any technology that makes wget'ing the site into raw HTML easy.

Unfortunately, the world of web development is so diverse in terms of both framework selection and the talent/experience pool, that it's very difficult to make a site that's easy to transfer to a new guy with an unknown skill set.

Have you had an experience like this? What did you do to resolve it? What lesson did you draw from it?


Posted by Erik | Permanent link

Nov 03

The latest web development stack

I've been using a new web development stack

    For web technologies:
  • Django - A nice framework. After some work, I've learned how to leverage URL's and templating, without the cruft of models, and without the admin module.
  • jQuery - dang, this is a time-saver. Just learn it.
    And for editing:
  • Vim - Omni-complete is perfect for web development - it closes my HTML tags and intelligently suggests CSS keys and appropriate values. Also, I've learned how to use the :mksession to preserve an editing layout.
  • ies4linux under Wine to test IE6 concurrently with Firefox.
  • dwm - I fought with Awesome 3 long enough to jump ship. dwm does everything I want, and nothing more.

Posted by Erik | Permanent link

Oct 26

On simple APIs

I wrote about my effort to learn sockets programming properly. For two weeks, every day, I wrote a simple socket server in C, from memory as much as possible.

It paid off this last week at work, when I finished a draft of a proxy server that sits between an existing server and a Flash client. A proxy is both a server and a client, so all my new skills were used, albeit in C# not C. The nice thing about sockets is that the API is fairly standard for all technologies - Unix and Windows have similar C API's, and all higher level languages simply wrap around them.

I'm so glad I learned the C first, since the standard API is lean. There are about 6 - 8 functions to learn to use.

Had I attacked this issue starting with C#, I'm certain I would have failed. The "user-friendly" wrappers contain a lot of distracting material. Furthermore, the MSDN documentation for class members always includes a program that does eight things, instead of being distilled down to the one thing I want to do.

So, here's to the distillation of problems down to their simplest atoms. I'll take bind(), accept(), and send() over SocketInformation, SocketInformationOptions, SocketFlags, and SocketOptionName any day.


Posted by Erik | Permanent link

Oct 19

Maker Faire 2008

How much fun is the Maker Faire? My 5-year-old had as much fun as I did, making it a perfect father-daughter geek outing.

I missed the Arduino talk, which is doubly-sad since I really need a kick in the pants to take mine out of the box and use it.


Robot carnage


Tesla coils play the "Dr. Who" theme. Inhaling the ozone in the top row, my kiddo said "It smells sweet, Daddy." Good girl.


Saying hi to R2-D2


Decorating a Day-of-the-Dead head


A self-amplifying art-bike adorned with music box players


The "Austin Bike Zoo" had its full menagerie


The rattlesnake bike


Posted by Erik | Permanent link

Oct 12

Writing portable apps under Gentoo

For a previous C project for work, I developed a Windows app on Linux. The clunky workflow:

  • Write the entire app in Linux, being careful to use only the C Standard Library.
  • Move the source onto a Windows box.
  • Using Eclipse,CDT, and MinGW, coerce a working build.
  • If cross-platform changes were needed, make the edits twice.

You can imagine this workflow is unsustainable for projects larger than my little console app. Fortunately, I'm now set up to do the whole shebang on my Gentoo box, using MinGW and Wine. Tonight, I've compiled and run a Windows-specific app completely from Linux.

Gentoo makes this setup easy, although it's probably a bigger production than it needs to be. I simply followed Gentoo's MinGW HOWTO to the letter, and had no issues.

By big production, I mean that instead of providing a quick-and-dirty Windows-specific solution, they've gone after the entire problem of cross-compiling for any architecture, with a single, enormous "CrossDev platform". One day, I'm sure I'll see the benefit of this framework. Among other things, it's meant to simplify the cross-compilation of dependencies (say, a UI library).

I haven't yet accomplished a full Autotools build that can compile a Linux or Windows app based on a configure script flag. It's high on my list, to be sure.

I'm completely stoked at the idea of compiling native apps for all major platforms. I don't want to be religious about portability. I just know that some programming problems aren't that hard if you plan the solution from the start.

Any cross-compiling devs out there? What experiences have you had?


Posted by Erik | Permanent link