Category Archives: annecdote

really, windows? 2014, and you do this to me?

Posted on by
1

 

the most depressing thing to happen to me today, looked a lot like this:

images

i was holding my iphone at the time.

the anachronism stayed with me for most of the day.

 

don’t even get me started on the sheer silliness of permanent deletion in an age of near-zero-cost storage. who permanently deletes anything anymore?

 

check this out, for an even more stark relief.

i think that, in an average year, i spend about as much time dealing with phone apps (100k loc or less) as i do dealing with a random windows version (40m loc). i’ll let you figure out which of the two category of experiences is more rewarding and/or pleasant.

whyfore chat, with django, twisted and websockets?

Aside

upon seeing the work i’ve put into writing tutorials, showing how to get realtime chat working in django + twisted/websockets, you might make the assumption that i consider this architecture to be, in general, a good idea.

A.

twisted’s implementation of websockets is, as of this writing, not integrated into the main branch.

don’t use code that isn’t considered, by its authors, to be reliable enough to merge into and release as part of their application distribution.

B.

twisted is an event-driven networking engine
django is a solid, easy to use web framework
websockets, a tcp based protocol, is usually implemented as a strange mix between the tcp and http protocols

it is, generally speaking, not a good idea to mix abstraction levels; adding event-driven components to your application by combining twisted and django is a bad architectural decision. I strongly suggest you consider using twisted.web instead of mixing django and twisted.

websockets are a strange mix of protocols, and can be difficult to work with unless you are very careful with your choice of libraries and application design, scope and implementation. at the time of this post, i would recommend against using websockets, in production, with the standard deployment of twisted. i strongly urge you to consider the following alternatives, in rough order of likelihood to work for you:

 

wtf javascript

Aside

javascript riddle time.

say that you’ve ventured into the land of XmlHTTPRequest ( $.ajax calls for the jquery people), and you’ve heard through the grapevine that (gasp!) it may be possible to make cross-domain http calls with this tool.

excitement! if you’re like me, you’ll see a chance to hack in fixes for a myriad of small bits of javascript you have lying around. so, off you’ll go, merrily trying anything that might work – your initial code might even look something like this, if you’re used to jquery:

  request = $.ajax({
        type: "POST", // !? - request sent through as HTTP GET
        url: 'http://someotherdomain.com/,
        data: 'some data',
        async: true,
        cache: false,
        dataType: 'jsonp',
        crossDomain: true,
        xhrFields: {
	       withCredentials: true //share cookies across domains!
	},
  });
  request.abort(); // !? - does nothing

confused

ok, so, technically this is a wtf jquery, more so than a wtf javascript. ‘jsonp’ crossdomain requests aren’t currently supported, natively, by most browsers – a native implementation would take significant care to avoid being a security risk (i’m not entirely sure it’s possible, even). there is, though, a fairly generic hack for implementing them. take a look at jquery-jsonp, which is close to the state of the art of what’s possible nowadays, and you’ll even see it there:

  // Create the script tag
  script = $( STR_SCRIPT_TAG )[ 0 ];
  script.id = STR_JQUERY_JSONP + count++;
  
  ...

  // Set source
  script.src = url;

create a new script tag, with the source set to be the target of your jsonp request, dynamically add it to the document tree, and wrap it in event handlers to handle the response or any errors.

since it’s a html script tag, it always leads to a browser HTTP GET request; and, since the request is initiated indirectly, by adding the tag to the document tree, some of the usual niceties for XmlHTTPRequest aren’t available – eg, the ability to interrupt.

mystery demystified – the behaviour is correct, though still unpleasantly surprising to me

[More programming riddles]

Apple vs. downloaded code: quarantined libraries

Posted on by
Reply

i do the majority of my work on OSX based machines – which means that i consistently run into a problem that looks like this:

osx file permissions

the context always throws me for a bit of a loop – usually, i’ve just downloaded some sort of library, or tool, file, and i can’t seem to be able to use it as i intend to. today, for example, i’d downloaded a javascript library and tried serving it via the osx version of apache, as part of an ongoing project – leading to strange looking 404 errors.

almost always (today is no exception), i find hints suggesting some sort of read permission related problem, and i eventually end up listing the file:

bash: ls -al some_file.ext

only to find that i do have read permissions, but the file has that pesky ‘@’ indicator (or, sometimes, a ‘+’ indicator) at the end.

so i’ve learned to associate the ‘@’ or ‘+’ symbol in a permission listing with ‘read permissions denied’, (despite knowing it’s actual meaning: the file has ‘extended permissions’, which is apple-speak for permissions which do not easily map to the unix-like permission system you’ll see at a bash shell)

at any rate, if you run into this kind of problem, there are two easy things you can do to get unstuck:

  1. use the finder to look at the file, and its access permissions – right click on the file, go to “Get Info”, and then expand the “Sharing and Permissions” tab. whatever problem you’re faced with, is almost always fixable from there.
  2. try finding the same info, only from the bash prompt – try running:
bash: xattr -d com.apple.FinderInfo

grinding from static design image to dynamic ui layout

Posted on by
Reply

say you’ve been given a static ui design – a gif, jpg, pdf, or other format file – and asked to create a pixel perfect equivalent version, in your dynamic application.

so you take a look at the design, and go back to the ui generating code of your application, and then start making small, incremental changes, gradually moving your ui components closer to the design – without breaking any of the underlying implementation and code interactions.

  1. make a small change.
  2. preview the built application ui.
  3. visually compare to the static file provided by your designer.
  4. how close are you?

in a previous post, i promised you an example of how i optimize the last step in this process. here is the example, in recipe format:

Setup: 

  • Mac/OS X system
  • modifying the ui for an iPhone application
  • design provided as a pdf file

Requirements:

  • Delta between the current state of the application design, and the static pdf
  • Direction for the delta
  • Pixel-level accuracy
  • No budget for expensive, related, design tools
  • No time to develop time consuming, related, design skills

Solution:

  • Gimp is a freely available graphic design/image manipulation tool, which runs smoothly on a mac. Install gimp
  • Gimp is scriptable – significant portions of the process can be automated. Automate as much of the process as possible, using script-fu (gimp’s macro functionality)
  • OS X comes with some powerful, built in, graphic manipulation components. Pay special close attention to the built in components (eg: screen capture) and the included tools (eg: Digital Color Meter)

For my specific example:

Preamble:

Command-Shift-4, then space, then click a window: Take a screenshot of a window and save it as a file on the desktop

  • ran the following command:
bash: defaults write com.apple.screencapture name "Screencap"

standardizing the file names generated by the screen capture utility.

Process for generating the UI:

  • Modify ui definition components in XCode.
  • Display the result in the iphone emulator/tab to the emulator
  • Screen shot of the emulator, only (Command-Shift-4, then space, then click a window)
  • Tab to a command prompt open on desktop and run:
bash: /Applications/Gimp.app/Contents/MacOS/gimp -b '(script-fu-overlay "Design_file.pdf" "Screen Shot 2013-09-17 at 7.04.32 PM.png")'
  • Visually inspect the two super-imposed images.
  • Delete “Screen Shot 2013-09-17 at 7.04.32 PM.png”, and start over

note that i actually automated a bit more of the process – starting gimp and deleting the image afterwards is a simple shell script, for example. the pattern, and many of the steps, have come in handy for many other similar situations – most recently, when working on an automatic pdf report generator

pixel perfect layouts

Posted on by
Reply

tl; dr

  • it’s difficult to get “pixel perfect design”, especially if your design work is outsourced
  • you’ll often see developers fighting to create pixel-perfect copies of designs provided as static images, inside of a dynamic, interactive, application
  • quick, effective, feedback, can significantly speed up the process
  • here’s an example where I turn “screen scan” and “overlay” into a one-click process

a good sense of taste is critical, especially in the current world of smart-phone centric startups. this is a particularly relevant meme to the startup world, and i often hear of startups and/or entrepreneurs which exhibit a good sense of taste being referred to as having, or being able to produce “pixel-perfect design”.

the original phrase “pixel perfect design”, in the design context, describes a very technical concept. it applies to human-produced images which display crisply on digital screens. specifically, for an image to be crisply displayed, it’s crucial that the contents of each and every pixel that is to be displayed, is accurately described by the designer; otherwise, screen software/hardware will take a best guess, usually by interpolating between nearby, better defined, pixels. the best guess/interpolation process leads to various, often visually distracting, side effects (usually unintentional blurriness in images). since modern screens have very high resolution (meaning there’s a large numbers of pixels to keep track of), and, especially when it comes to user-interfaces, most designs are displayed on more than one resolution – the process of producing pixel perfect designs requires specialized skills and often slow, pain-staking work.

in the startup world, the phrase is nowadays used to refer to a more general concept – i.e., the combination of good taste with the hard, painstaking work required to produce good quality, tasteful, designs – for user interfaces and elsewhere.

very frequently, startup founders don’t have the required skills to turn their sense of taste and ideas into crisp on-screen designs. it’s also unusual for a startup to have a designer on-staff, especially in the early stages, where product and idea development takes the bulk of the effort at the startup, and there may not be much to design, most of the time. design work at these stages is almost always outsourced

this leads to repeated pattern – a developer at a startup finds himself with the task of recreating, in a dynamic user interface, a painstakingly produced design, provided someone else in a static image format

many developers lack the skills, and tools, to quickly and efficiently do this kind of work. they won’t have access to often expensive design tools, such as those provided by adobe, and may not even know which tools would be useful and how to apply them. surprisingly often, though, an investment of time spent upfront, combined with some easy to find free tools and basic scripting skills most developers do have, can save significant pain and work

you don’t have to take my word for it. since i usually find it especially useful to automate the feedback portion of the work, i thought i’d provide an example – see here

When is a javascript function *not* an object?

Posted on by
Reply

yesterday, i was in the middle of testing an optimization to a javascript tool, when i saw this sequence occur in my debugger:

o // function (a1, a2, a3) {  return a1 + a2 + a3; }

typeof(o) //'function' (1)

for (var n in o) { console.log(n); } // --> a long list of items

Object.keys(o) //TypeError: not an object (2)

Object.prototype.toString.call(o); //"[object Function]"

headscratch

clearly there’s a bug here, somewhere. but where? care to take a guess before reading on?

instinctually, this felt like a deep kind of problem. maybe a vm level bug, or a strange corner case in the javascript language definition.

as far as a bug goes, though, these kinds of bugs, are rare. Object.keys is a core function – significant production level code uses it, and relies on it functioning properly; it’s incredibly unlikely that a bug, especially one this major, would have escaped into a production level browser.

and if this strange behaviour is part of the ECMAscript standard, then it would definitely be near the top of javascript gotchas – i would have seen it documented somewhere.

so, i reasoned, this must be a result of work done in the particular bit of java code i was looking at. someone must be explicitly messing around with the system, and there should be traces of the relevant javascript in my current debugging session, not anywhere deeper down in the system.

unfortunately for me, despite having reasoned correctly thus far, a quick check for code that might alter the systems’ behaviour didn’t yield results. so i followed my instincts, and proceeded to, in parallel, test for a bug in the virtual machine, at the same time reading through ECMAScript specifications. time not completely wasted – i learned, for example, that:

  • the ECMAScript definition is not fully clear on certain components – such as “host object”s. as far as i understand, these are convenience objects the vm may provide to a user, and are usually vm dependant. so while there may be consensus between browser vendors on some of the objects, specifics vary between vendors, and implementations may not be complete, or standard compliant. not entirely relevant here – my object was a standard, run of the mill, javascript object, and not a “host object”. still interesting though
  • this kind of issue may pop up if you’re dealing with concurrent code execution – an object can mutate under your feet, for example. very unlikely for most javascript applications, though, and not an issue here
  • the output of native functions, like Object.keys and Object.prototype.toString.call, can vary depending on how an object is instantiated (try regular var a = function.. versus var a = new Function(…)). the function i was looking at, however, didn’t do any strange instantiation behaviour

my instinct, of course, was off – and my initial reasoning had been correct.

after much experimentation, i eventually (almost accidentally, since i wasn’t explicitly looking for it) discovered a dynamically loaded bit of javascript, which was (drumroll) …

overriding the native implementation of Object.keys, with a buggy bit of code.

so, false alarm. javascript functions are, as expected, always objects, and my VM doesn’t include a very nasty looking bug.

pickling python objects

Posted on by
Reply

seethesource talks about storing and loading python dictionaries from disk. his sample code was easy to read, but, after some thinking, seems incomplete (for example, the storing function assumes we’re only storing string elements to disk; there are likely unicode formatting related problems for strings; etc…).

(in my efforts to level up my python skills, i’ve been scanning blogs for python problems and solutions, to see what i can learn – i wrote this post in that context)

saving/loading objects to/from disk sounds like a very common problem to solve; python should have a library for something this common. so, before trying to write my own, more robust, version of seethesource’s code, i dug up the python documentation wiki. i was rewarded with a lesson on object pickling (pickling is pythonic for serialization?). the wiki even included a handy, relevant, example:

# Save a dictionary into a pickle file.
import pickle

favorite_color = { "lion": "yellow", "kitty": "red" }

pickle.dump( favorite_color, open( "save.p", "wb" ) )

# Load the dictionary back from the pickle file.
import pickle

favorite_color = pickle.load( open( "save.p", "rb" ) )
# favorite_color is now { "lion": "yellow", "kitty": "red" }

recursive touch

Aside

i just moved a bunch of media files from one device, to the other, and the file creation dates for a small number of my directories (and their contents) moved a few years into the future.

this problem smelled like a python 1 liner; i came pretty close – highlight to see the code:


import os, sys, subprocess

for d,s,fs in os.walk(sys.argv[1]):
    for p in (s+fs):
        subprocess.call(["touch",os.path.join(d,p)])

(source online, in case I improve it)

[More programming riddles]