Digital Flapjack

The Digital Flapjack Blog RSS

PlaceWhisper on Digital Planet!

Posted by Michael Dales on 2010-07-23 20:40:32

Digital Planet, the BBC World Service's weekly technology program, is usually very interesting, but this week's episode was of particularly so - PlaceWhisper gets discussed briefly during an article on geo-located data!

We're totally stoked to be mentioned - quite an honour. We doff our hat to Bill Thompson, who raises some good points on data persistence in applications like PlaceWhisper, and how do you stop the world becoming littered over time with persistent spacial data. We've got some ideas on how PlaceWhisper will deal with this, and it'll be very interesting to see how these hold up to real world data over time.

Introducing PlaceWhisper!

Posted by Michael Dales on 2010-07-23 07:39:42

Remember when you were kid, you'd leave secret messages for your friends to find? Maybe you'd make trails for them to follow, perhaps leading to some treasure? Perhaps you'd leave a message hidden in a place where only your friends could find it? As you grow up you tend to find yourself without chalk to hand to scrawl a message in an out of the way corner, and leaving bits of paper under rocks for your friends to find is harder to do in the city. But wouldn't it be great though if we could still engage with people in that form? Leaving messages for friends or strangers, telling them that you love the plum cake at this cafe, that this is the spot where you first met your partner, or leaving a trail for people to follow that describes your favourite walk through the village or city?

That's what PlaceWhisper lets you do. With your mobile phone you can create and discover hidden messages, called Whispers, each tied to a specific physical location. PlaceWhisper lets you create Whisper, which are either shared with everyone how visits the place, or they can be hidden so that only your friends can find them. You can use PlaceWhisper to find Whispers where you are - nearby Whispers will show up on your phone, and when you walk up to the spot where they were created, their contents can be revealed! If you've ever done geocaching, then it's very similar to that, only without the Tupperware!

You can also join Whispers together to form a trail, that gradually reveals itself as people collect it, recreating the fun of a treasure hunt. With trails you can guide people around a city or the countryside without having to litter the physical world with signposts.

This is PlaceWhisper, a fun app that lets you interact with the world in a new way, and the applications are only limited by your imagination!

We've just released the first version of PlaceWhisper for the iPhone. Over the coming months we're going to be working hard at improving both the iPhone version, building versions for other mobile platforms, and creating a web front end that lets you find out who's been collecting your Whispers, create trail scoreboards, and all other manner of exciting stuff.

So, why not go give it a try? PlaceWhisper is available as a free download from the Apple App Store (UK only currently, releasing in other countries very shortly), and you can follow PlaceWhisper on Twitter: @placewhisper.

New look website!

Posted by Michael Dales on 2010-07-21 20:21:45

After looking the same for a couple of years, we decided it was time to overhaul the look of digitalflapjack.com, and you're looking at the result now (unless you're using an RSS reader, in which case head over right now and check it out!). The new look is to coincide with the launch of our fun new location based application, PlaceWhisper - more on which on the blog shortly!

Both the PlaceWhisper artwork and the initial design for the new digitalflapjack.com were produced by the amazing Andy Field - Andy did a great job of capturing the fun spirit of what we're doing at Digital Flapjack. We're very pleased with the results at Flapjack Manor, we hope you are too.

This refresh is just the start, we've got a lot more to do still, both with this site, and with exciting developments as we continue to work on PlaceWhisper. Stay tuned!

Parallelisation made easy: CoreData and worker threads

Posted by Michael Dales on 2010-06-11 10:18:35

Recently I've been working on an iPhone app that does a reasonable amount of CoreData munging, which was causing my UI to lock up. After trying to optimise my use of CoreData, I managed to improve things, but still was having UI slowdown on things like the first every sync, where lots of data comes in, so I moved that processing to another thread. This is quite easy to do in Cocoa, but threading is something that tends to fill people with fear, so I thought I'd write up just how easy it is to do both the threading and make CoreData work well with multiple threads.

Let's start with some example code to set the scene:

- (void)processIncomingData: (ASIHTTPRequest*)request
{
    NSString *rawdata = [request responseString];
    NSArray* lotsOfData = [rawdata JSONValue];
    [request release];

    MyApplicationDelegate *delegate = [UIApplication sharedApplication];
    NSManagedObjectContext *context = [delegate managedObjectContext];

    for (NSDictionary* info in lotsOfData)
    {
        // create or update managed object here
        // probably some searching too
    }
    NSError *error = nil;
    [context save: *error];
    if (error)
        NSLog(@"error %@", [error localizedDescription]);

    // let the app know we're done
    [self reloadInfoFromCoreData];
}

So here we're using ASIHTTPRequet to get some JSON data, we then have an array of dictionaries that we're going to store in the CoreData store. As part of that we're probably going to check whether the objects already exist and update them if necessary, otherwise make new objects. Finally we want to update any UI representation of the data once we've loaded all the data. It's this loop and then the save that's potentially slow, and thus something we want to move to another thread, so as not to block our main thread of execution, which will lock up the UI.

Threading is typically a messy business. When I was a researcher at Intel many years ago, one of the things that we worried about was that multicore was on the horizon, and raw thread programming is difficult and error prone, and most programmers can't cope with it. When I saw NSOperationQueue appear in Cocoa I was hugely relieved - whilst you still have issues with shared data, here was at least a good model to deal with the messy world of creating. managing, and destroying threads. In fact, it's so easy, we can take our code above and parallelise it really easily.

NSOperationQueue is an object that managed a set of worker threads, and you simply give it jobs to do, and it'll queue them up until a worker thread can do the task. You don't need to worry about creating these threads, how many there are, etc. NSOperationQueue manages all that. You can just worry about giving it work and letting it deal with it. So let's do just that.

The first thing we need to do is create an operation queue. I do this lazily in my viewDidLoad method in my UIViewController for my iPhone application:

@interface MyViewController : UIViewController
{
    NSOperationQueue *operationQueue;
}

- (void)viewDidLoad
{
    if (operationQueue == nil)
        operationQueue = [[NSOperationQueue alloc] init];

    [super viewDidLoad];
}

And that's all that's required to get set up (you should remember of course to release the queue too when you're done with it). Now we're ready to take our method above and move it to threads. Let's assume we'll move the heavy lifting of our processIncomingData method above into a new method that will just tod the CoreData work called storeDataInCoreData, which we'll define later. It's this method we'll want to call in another thread. To do that we change the processing method to look like this:

- (void)processIncomingData: (ASIHTTPRequest*)request
{
    NSString *rawdata = [request responseString];
    NSArray* lotsOfData = [rawdata JSONValue];
    [request release];

    NSInvocationOperation *operation;
    operation = [[NSInvocationOperation alloc] initWithTarget: self
                                                     selector: @selector(storeDataInCoreData:)
                                                       object: lotsOfData];
    [operationQueue addOperation: operation];
}

Again, that's it - nice and clean and simple. We build an invocation operation which will run the specified method on a given object and pass that to our operation queue. This will now happen at some point in another thread, and we didn't have to do much at all. Now we just need to look at the way we use CoreData in our new thread, so let's take a look at storeDataInCoreData.

If you use the standard iPhone CoreData project templates, you'll be used to asking the UIApplication delegate for the managed object context. This is a really handy way to save you creating many contexts, particularly as the context also acts as a sort of cache for managed objects and data. Unfortunately you can't share contexts across threads, as they don't support concurrent access, we'll need to create a new NSManagedObjectContext for our thread to use. Thankfully, that's not too difficult.

The other thing we want to do is tell our view context to update any views on the data, which we did originally by calling a self definied reloadInfoFromCoreData method. We still want to do this, but we need to call it back on the main thread, as it's probably going to mangle data structures that the UI uses, and we don't want to worry about locking any of that, so we'll just let the main thread still handle the data reload. Thankfully it's easy to call back to the main thread from the worker thread.

Thus, we end up with:

- (void)storeDataInCoreData: (NSArray*)lotsOfData
{
    // create our new context
    MyApplicationDelegate *delegate = [UIApplication sharedApplication];
    NSManagedObjectContext *context = [[NSManagedObjectContext alloc] init];
    [context setPersistentStoreCoordinator: [delegate persistentStoreCoordinator]];

    // do the work
    for (NSDictionary* info in lotsOfData)
    {
        // create or update managed object here
        // probably some searching too
    }
    NSError *error = nil;
    [context save: *error];
    if (error)
        NSLog(@"error %@", [error localizedDescription]);

    // free up our context
    [context release];

    // let the app know we're done
    [self performSelectorOnMainThread: @selector(reloadInfoFromCoreData)
                           withObject: nil
                        waitUntilDone: YES];
}

Here you can see that the method is slightly longer than in the single threaded case it would be, but nothing too complicated. We create a new managed object context over the same CoreData store, use that for all our CoreData operations, then release it once done. After that we call back to the main thread to tell it it can now refresh its data from the CoreData store. And here comes in the last detail you need.

If you recall we said one of the nice things about NSManagedObjectContext was that it cached data to save you time. So if you've already loaded data from a particular table in the CoreData store, althought you've added or updated all this new data in the worker thread, the main thread by default won't see it, as there's no timeout on the cached data in the application delegates managed object context. We need to force a cache flush before the view controller will see the updated data.

In my application I have a root object that has relationships to all the rest of the data: my user object then has relationships to all the data they're interested in. So what I want to do is force a reload on that, and as I traverse relationships they'll all fault and load in new data. So in the top of my reloadInfoFromCoreData method I simply need to do:

    MyApplicationDelegate *delegate = [UIApplication sharedApplication];
    NSManagedObjectContext *context = [delegate managedObjectContext];
    [context refreshObject: myRootManagedObject
              mergeChanges: NO];

After this, any data I retrieve via this root object will be refetched from the store.

And that's all there is to it. We've managed with relatively little code and little complexity to move our CoreData processing to a child thread, meaning we can process lots of data without our UI becoming unresponsive. There's lots more you can do with NSOperationQueue, and I recommend you investigate it to see how it can help you defer batch processing from your main thread where possible. It's a small overhead to your code, but guarantees a much better user experience if you're not locking up your UI all the time.

Avoiding excessive interdependency with UIViewController

Posted by Michael Dales on 2010-05-16 16:53:31

I'm a big fan of the Cocoa frameworks in general, which typically have a very clean interface. However, in Cocoa Touch there's one glaring failing with interfaces that be default cause an annoying amount of class specific knowledge leaking out into other classes, and that's the default constructors for UIViewController and UITableViewController.

By default, when you create a new UIViewController or UITableViewController in XCode it will have a constructor that requires you either provide the name of the nib file to use or the table style to use. Now the code that creates the new view controller needs to know how it's going to look - surely that detail should be taken care of inside the view controller class itself? So much for abstraction!

My solution to this, and one that I've seen others use, is to create my own default construction that simply hides these details. So for a UIViewController I definite init as such:

- (id)init
{
    return [super initWithNibName: NSStringFromClass([self class])
                                bundle: nil]) != nil);
}

You can specify some other name for the Nib, but the typical case in XCode is that your xib file will have the same name as your class. For a UITableViewController it's even simpler:

- (id)init 
{
    return [super initWithStyle: UITableViewStylePlain];    
}

Now, no matter what type of UIViewController subclass I'm creating, I can simply just do something like:

   MyViewController *controller = [[MyViewController alloc] init];

And thus you've removed a painful dependancy between your classes. Not rocket science, but this little tip should really be something XCode does for you.

Making CoreData slicker with mogenerator

Posted by Michael Dales on 2010-03-26 12:16:42

After returning to Cocoa development after a long absence, it was great to discover all the new frameworks there are now to help you get your apps together quickly. After getting used to ORMs as a way of abstracting away databases whilst doing web development, I was very pleased to get to grips with CoreData, Cocoa's abstraction over the sqlite database.

CoreData has lots of nice features. Despite being a coder, I like the way I design with my data model in the model editor outside of code, then work with managed objects in my code - it's a boundary that seems to work well. XCode helps make this relationship work by auto generating (on request) code for custom managed object subclasses for each table in your data model.

Unfortunately, these code files are hard to maintain, due to the usual dilemma of auto generated code. If you change your model you're faced with two choices: update the files yourself, laborious but does mean you can add logic code to these subclasses without fear of losing it, or you can have XCode regenerate them to reflect the changes, but you can't really keep any logic code in those files. Another slight annoyance is that these subclasses only really save you from using key/value accessing managed objects - they don't save you from a lot of boiler plate code that you end up writing with CoreData to do with queries and creating new objects.

But there is a solution, thanks to Jonathan 'Wolf' Rentzsch, called mogenerator. mogenerator is like XCode's subclass generator, but on steroids. Firstly, it generates two classes per table. The first is a direct subclass of NSManagedObject and adds accessor methods for attributes and relationships specific for this table, just like XCode. However, it also generates a second subclass, which is a subclass of the first one, which is the one you use and can edit freely, as mogenerator will only ever update the first subclasses's files after creation. Thus you can add your own table specific login to this file without ever worrying it'll be lost when mogenerator is run again after you've updated your CoreData model.

The second very useful thing mogenerator does is it creates more than just direct accessor methods, it adds a bunch of useful calls that help reduce the amount of boilerplate code you need to write. For example, there's a method for creating a new managed object instance without having to also remember the CoreData table name - not having this is clearly an opportunity for mistakes to creep in (and I've just submitted a patch that will also do the same for making entity descriptions this way). It also adds hugely useful value accessors for numeric values, so in addition to getting the NSNumber object, you can also get and set using integers or floats as appropriate - another useful time saver.

You could do most of what mogenerator does yourself, but why should you? mogenerator lets you just get on with coding. If all this wasn't enough, if you install mogenerator it comes with a tool that, if you set a comment "xmod" on your CoreData model (under GetInfo in XCode) then mogenerator will automatically be ran any time you save changes to the model, so your code is always up to date in reference to the model. It really is a must if you're playing with CoreData.

EmberExporter for Aperture 1.1 released

Posted by Michael Dales on 2010-03-17 08:23:42

We've just released an update to EmberExporter for Aperture, bringing it up to 1.1. The major change here is that the 1.1 release is fully compatible with Aperture 3, adding 64 bit compatibility. This means Aperture 3 users no longer need to relaunch Aperture in 32 bit mode when using EmberExporter!

Otherwise, everything else is pretty much the same - EmberExporter is the easiest ways to get your photos from Aperture and onto Ember so you can share them with the world!

Existing users should download the latest release and install it to get the new Aperture 3 support, or hope over to the EmberExporter page to learn more about what it does.

WhereCampEU and DFGeoContext

Posted by Michael Dales on 2010-03-16 20:33:03

Last weekend I had a good time at the first run of WhereCampEU, a two day unconference dedicated to all things location based. I had a good time there, meeting new people, and learning new things. In an attempt to contribute back, I gave a short talk of on how to write my first iPhone location app:

My first iPhone location app hosted by Ember

There were lots going on about maps and visualisation, all very cool and interesting, but I'm also very interested in the indirect use of location to aid the user experience in applications. A lot of people use their laptop or mobile phone for both home use and work use, yet neither device acknowledges this duality. What sort of opportunities are their for enhancing the user experience when your application knows the geographic context in which decisions are made and data is entered? Imagine an IM client that knew which people you talked to at work, and which at home, and displayed those users appropriate to where you were at a give point in time. Imagine a browser that notes where you commonly use certain bookmarks, so reorders them to make work related ones more prominent when you're at work, and less so when you're at home.

The mad thing I realised when prepping sample code for my iPhone talk, is that all the APIs you need to do this are available on not only the iPhone, but also in Mac OS X, so there's no reason we can't implement this sort of behaviour today! So, for the hacking competition ran on the second day of WhereCamp EU, I set about to make an easy to use library that lets your application find out with next to no effort, and even more importantly at no extra effort at all from the user, where the user is: at home? at work? or somewhere else? I give you DFGeoContext.

An overview

DFGeoContext's interface is simple. You create a context object, and set yourself as the delegate for location context updates:

    self.userContext = [[DFGeoContext alloc] init];
    userContext.delegate = self;

Then, you simply tell it when you want it to start looking for context information:

    [userContext startFindUserContext];

Then, you implement the delegate method that DFGeoContext will call when it's found the user's context:

- (void)userContext: (DFGeoContext*)context
          updatedTo: (NSString*)location
{
    [displayThingy setStringValue: location];
}

The location value should be one of the following values:

  • kDFGeoContextHome
  • kDFGeoContextWork
  • kDFGeoContextOtherWhere
  • kDFGeoContextUnknown

These indicate it the user is at home, at work, some location other, or the service could not work out the current location.

And that's all there is to it!

From the user's perspective, they don't need to do anything. Your application doesn't need any new control or such - all the information needed to make this work is already there - that's the beauty of it. Most users will already have entered, either directly or indirectly on setup, their home address into the system AddressBook. If you're lucky, they'll have added their work details too (assuming they have a work address). DFGeoContext takes advantage of all this existing information so you don't need to pester the user for it.

How it works

Under the hood, DFGeoContext simply pulls together three existing APIs:

  • CoreLocation
  • AddressBook
  • CloudMade's Geocoding service

The first two are native libraries on the Mac and iPhone. CoreLocation lets you get the latitude and longitude coordinates of the device, so we can work out where we are currently, clearly essential. Although everyone knows about CoreLocation on the iPhone, it's actually available on Mac OS X too since Snow Leopard, where it uses local wifi base stations to estimate location. The AddressBook has a very simple API to let applications inspect the user's information, including things like home address and work address.

At this point, it sounds like we're done, but not quite. We need to convert the textual addresses for home and work into latitude and longitude so we can compare them with the information from CoreLocation. Unfortunately not even the iPhone has an API to do forward geocoding - the ability to turn a textual name of a place into coordinates. For that we need to use a third party web service, and for that I've picked to use CloudMade, the commercial friendly API side of OpenStreetMap. What I like about CloudMade, as opposed to say Google and all the other APIs that can also do this is that they've got nice support for both open projects and commercial projects, so you can use DFGoeContext in either. You just need a key, and the commercial rates for API use are actually documented and reasonable, unlike some other companies I've looked at.

Anyway, so we call CloudMade with our addresses and get back coordinates, and we're done. We can then work out if we're at any of the locations in the user's AddressBook, or we're somewhere else. To make getting the demo together easier, I've used the excellent ASIHTTPRequest library for API calls to CloudMade, and the SBJSON framework for decoding the JSON data CloudMade return.

It's a winner!

Having hacked this together in an afternoon, I didn't have time to put a slick demo app together - I presented this along with a very simple dialog box interface to test it. However, the audience at WhereCamp EU clearly got the idea, as DFGeoContext won the hacking contest! There was only a small number of entries, but the other entrants made very good demos, so it was quite an honour to win.

There's lots still to do thought with DFGeoContext. I'm really pleased with it in one sense - I've got a library here that requires no effort from the user for applications to start doing sensible things in relation to the context they're in - that's something that excites me, and I look forward to trying to user it in future applications. However, there's some bits that need tidying up still:

  • CoreLocation on the Mac seems a little temperamental. I need to experiment more with this to see how reliable it is. Worked fine when we were in London, but less so in the sticks where I live.
  • Add caching. At the moment it makes too many calls to CloudMade, where as a little caching to a preferences file would fix this.
  • Add more sanity checking on the data from CloudMade. For this hack I knew what to expect, but the decoding is a bit hardcoded at the moment.

But still, not bad work for an afternoon's playing with some APIs. The code is available under an MIT style license on github. It's still a little rough, and I'll tidy it up as I get chance, but I welcome suggestions and feedback on it!

Finally, just a word of thanks to the people who put WhereCamp EU together - well done! It went off smoothly, and had lots of great people and presentations. Looking forward to the next one!

EmberExporter for Aperture released!

Posted by Michael Dales on 2010-02-24 18:00:19

Digital Flapjack is pleased to announce version 1.0 of EmberExporter for Aperture!

We've long been huge fans of Ember, the online design scrapbook by Realmac Software, and we're also huge fans of Apple's Aperture photo management software, but we've been frustrated that there was no way to directly get images from Aperture to Ember. Until now!

EmberExporter is an Aperture plugin that lets you select one or more photos in Aperture, and then export them directly to Ember. Along the way you can modify attributes like captions, privacy settings, and tags, so that your images are as you want them the moment they arrive on Ember.

EmberExporter for Aperture is available now for just $10. You can also download a trial version to test it out before you buy (the trial version isn't time limited, but will watermark any uploads).

EmberExporter is compatible with both Aperture 2 and 3. However, as Aperture 3 was just released as EmberExporter was in the final stages of development, there is no 64 bit support for Aperture 3 yet - you'll need to switch Aperture 3 to 32 bit more to use EmberExporter. We plan to address that in the future with a 1.1 release.

Upcoming things

Posted by Michael Dales on 2010-02-14 12:13:25

We've got multiple things on the bake at the moment, but one little project hatched out of our trip to NSConference, and bumping into those nice chaps from Realmac Software. It's in testing now, and we'll be sure to shout about it when it's out, which should be Real Soon Now :) For now, you can take a sneak peak at the work in progress screenshot :).

NSConference 2010 Photos

Posted by Michael Dales on 2010-02-04 18:24:41

I've spent the previous three days at NSConference 2010, a Mac and iPhone development conference. To call it a conference is a bit of a misnomer, as it's as much about getting the community together over coffee and beers as it is about learning during talks.

It was a total blast - I learned lots and got to meet lots of interesting people and old friends. I plan on writing up some more detailed thoughts, but that'll take a little longer. But for now I've just uploaded all the pictures I took whilst at the event.

Making the web easier in Cocoa

Posted by Michael Dales on 2010-01-20 17:56:43

Having come back to Cocoa in this last year from having spent a few years writing Python, I’ve taken a much more holistic approach to writing Mac and iPhone code. Where as before, having come from being a C hacker, I was writing lots of low level code myself (and to be fair, six or seven years ago, or whenever it was I wrote things like osx2x, there wasn’t a huge alternative) now I find myself looking around to see what libraries exist to help me get on with the interesting bits. This is one of the really great things about Python – it’s library support, both in terms of the standard libraries and what’s out there in the internets, is fantastic, and now the same applies for Objective C.

So, I thought I should post about a couple of the more useful ones I’ve been using recently. As a full on webhead these days, a lot of the client code I write is to interact with web services (as Nik Fletcher said recently, it seems like writing a Twitter client is the new “Hello, World!”). I’ve been using two libraries to help me here: one to make the requests, and the second to help decode them.

First up is ASIHTTPRequest from All-Seeing Interactive. This library makes it really easy to start firing off requests to web services without having to worry too much about the code. It comes with very good documentation with examples, and has lots of advanced stuff I’m yet to take advantage of, such as automatically updating progress indicators in your UI, talking to S3, etc. I’ve been playing with the form request functionality, which makes it very easy to do an authenticated POST of data to a web service as part of an API. If you’re talking to the web and you’re using Cocoa’s standard libraries for network interactions, you really should give this a look.

The second recommendation is json-framework. JSON is becoming a fairly common marshalling language for web APIs to talk – it’s less verbose than XML, it’s still human readable, and it’s easy to code with. The JSON framework simply adds categories to NSString to generate NSArray or NSDictionary objects from strings containing JSON, and to NSArray and NSDictionary to generate an NSString containing the JSON encoded data. Very easy to use, and I’ve used this one lots with no issues.

There you have it, two very useful libraries that work well on both Mac and iPhone, that I highly recommend if you’re doing any web API work.