Monday 24 September 2012

Intel Ultimate Coder Challenge - Part Six

Journey’s End

Well boys and girls, I hope you enjoyed our adventure into the mind of a software coder and hopefully gleamed some useful insights into the world of Ultrabook development! From a five page design document to a finished Ultrabook app, it’s been a labour of love through a mine-field of cutting edge technology.

Reflecting on the journey, my decision to create an entire WinRT engine in six weeks was simply bonkers. I don’t recommend it to anyone who values their sleep!

As much as you might expect this blog to be a nice neat (and short) wrap up of the project, I spent every day of the allotted six weeks coding my little socks off to ensure I could get as much into the final app as possible. I have something for everyone in my final episode, including a ten minute video of the Love Hearts(r) app and a gallery of screen shots.

I also want to cover the technology I implemented in my final week, including in-app purchasing, Ultrabook camera and social integration. Before we charge headlong into the technology, let's see the final Love Hearts(r) app in action:


I'll be the first to agree my video recording skills are less than passable, so I have created a wall of images which explodes the app so you can see each screen at a glance in a little more detail:


An app crammed with sensor goodness
can you find them all?

You can send custom photo messages through
all your favourite social outlets


Maintain a friends list to make sending
more messages a breeze

Type your message using the Ultrabook keyboard,
no need for a virtual one

Use the built-in Ultrabook camera,
or access the Photo Library
  
Position and crop the capture
for that perfect wave

Take your photo into the art tool,
ready to customise your picture

Use coloured pencils and brushes, plus rubber
stamps for a beauty make-over
  
Swipe the screen to reveal a
new surprise from the app

One surprise might be a piece from a jigsaw,
can you tell what it is yet?

Another activity is the Love Train

Can you beat your friends highest score?
  
Discover love poems, and send them
to your friends and loved ones

A red star notifies you of unopened
presents within the app

You are also notified of unopened presents
from the Windows 8 tile page

One of the surprise items might
be a joke you can share

You can also send messages to
another Love Hearts app

A bubble appears when a message
is waiting for you

Windows 8 tile page also lets you know
when a message is waiting
  
Tapping the bubble opens the message,
this one is a joke someone sent me

The joke is added to the joke heart at the
top of the page, just tap to laugh

The app monetises through the Windows 8
in-app purchase system

When published to the Windows Store, the app will
give you a price to buy unlimited credits

Find a love compass, always pointing to
your true love (providing it's north) 

Create photos and then share them
through your social graph

The app re-creates your message in the cloud
and allows you to tweet the link
  
You can also send the link as a Facebook update

Clicking the link takes you to your message,
courtesy of the cloud
For more information about app availability, I will continue running this blog site so check back in a few weeks to find out which stores you can download it from.

Windows Store 'Share' Button

The ‘Share’ option on the right slide-in bar enables apps to share content with other third party apps. The guidelines suggest that most if not all Windows 8 apps should take advantage of this feature which is expanded upon here: http://msdn.microsoft.com/en-us/library/windows/apps/hh465251.aspx

I was intending to add this feature but soon realised that optimal use of the feature would need some design changes to the app. For example, the current app asks you to click the Twitter icon, and then compose a message and finally takes you to the browser to enter your twitter account details and post the message link.  With the new Windows 8 ‘Share’ feature, the app could have been designed to have the Message Editor independent of the delivery method. When a message had been created, the user could simply swipe from the right, press Share and select Twitter, Facebook, Email or Browser. It's an elegant feature, and one I shall be taking full advantage of in the future.

Monetising an Ultrabook App

Initially I thought the Windows 8 Store did not allow in-app purchasing, but on closer inspection I discovered that it actually supports multiple forms of monetisation and enough documentation for me to get started supporting this under WinRT.

A great place to start the learning, and the starting point of my own mini-adventure was here: http://msdn.microsoft.com/en-us/library/windows/apps/br230836

With less than a week to go, I figured it made sense to register my particulars for a Windows Store developer account which lead me to the part where I had to pay £37 and fill in some rather aggressive online tax forms. This would give me the back-end ingredients I needed to create in-app tokens that represent what I chose to sell from within the app.

Finally, I located a C++ sample which in theory would contain all the code I needed to get in-app purchasing off the ground, found here: http://code.msdn.microsoft.com/windowsapps/Licensing-API-Sample-19712f1a

I did discover when creating in-app tokens that Windows Store does not allow internal currency as such, instead opting to sell in-app features which last a set time from 1 day to forever. As I needed a ‘buy credits’ feature, I set the ‘More Love Credits’ token for one day and would use some internal code to store app credits manually.

The first integration step was to create some in-app purchase commands for Freedom-Engine, and some code which would be handed over to Steve to add to the BUY button in the app:

rem Set the name of the app for Purchase Dialog
InAppPurchaseSetTitle ( "Love hearts" )

rem List out all in-app products and setup
InAppPurchaseAddProductID ( "More Love Credits" )
InAppPurchaseSetup()

rem If 'click', attempt to purchase the first item
if BUYButtonPressed=1
 InAppPurchaseActivate(0)
 while GetInAppPurchaseState()=0
  rem app is busy obtaining in-app feature
  Sync()
 endwhile
endif

rem Detect if first in-app product purchased
if GetInAppPurchaseAvailable(0)=1 and NoNewCreditsToday=1
 rem User has purchased 'More Love Credits'
 inc LOVECREDITS,50
 NoNewCreditsToday=0
Endif

Of course behind the scenes, this looks a little different in WinRT code, and after some initial wrangling managed to distil it down to some essential blocks of code. The first section is the classes we would be using:

using namespace Windows::ApplicationModel;
using namespace Windows::ApplicationModel::Store;
using namespace Windows::Storage;

Second, we are using the Store Simulator to test in-app purchases prior to publishing to the Windows Store, so we need to load in ‘pretend’ products, which is done with an XML file and the following code:

create_task(Package::Current->InstalledLocation->GetFolderAsync("data")).then([this](StorageFolder^ proxyDataFolder)
{
    create_task(proxyDataFolder->GetFileAsync("in-app-purchase.xml")).then([this](StorageFile^ proxyFile)
    {
create_task(CurrentAppSimulator::ReloadSimulatorAsync(proxyFile)).then([this]()
            {
        });
    });
});

Finally, the essential WinRT code to detect if a product has been purchased, and also to buy the product on demand is only a few lines:

auto licenseInformation = CurrentAppSimulator::LicenseInformation;
auto productLicense = licenseInformation->ProductLicenses->Lookup("product1");
if (!productLicense->IsActive)
{
create_task(CurrentAppSimulator::RequestProductPurchaseAsync("product1", false)).then([this](task<Platform::String^> currentTask)
     {
            try
            {
                currentTask.get();
                auto licenseInformation = CurrentAppSimulator::LicenseInformation;
                if (licenseInformation->ProductLicenses->Lookup("product1")->IsActive)
                {
                    // You bought Product 1.
                }
            }
            catch(Platform::Exception^ exception)
            {
                // Unable to buy Product 1.
            }
        });
}

As you can see, it’s relatively simple to monetise your app, and when you are ready to publish, simply substitute the CurrentAppSimulator with CurrentApp and you’re ready to go. For a working sample of in-app purchasing, Google ‘Trial app and in-app purchase sample’ for the example.

Submitting an Ultrabook App that uses DirectX and Webcams

Although there are a huge number of things to test when preparing your app, there a few gotchas that apply specifically to how you describe your app when submitting it. For example, you need to specify what the minimum DirectX feature level is for the app (9_3, 10_0 or ALL)? For Love Hearts, I am using a relatively simple shader which means I can choose ALL, but if you are for example making use of the special features of the Intel HD Graphics 3000/2000 Sandy Bridge chipset, then you need to specify 10_0 when submitting your app.

Also, if your app uses the webcam, which Love Hearts does, you cannot set the age rating to 3+ and the guidelines recommend you set this instead to 7+. Given my market, this was not an issue. The store also allowed rating board approvals to be optional for many target countries, which was a blessing for a small developer like me.

Windows, Drivers, Countryman!

A quick tip for developers who are working their way out of Windows 8 Release Preview to Windows 8 RTM, also check to make sure you are not using the Visual Studio Express 2012 RC which was only designed for the release preview OS. You can now find the official VS Express download from the Microsoft website. I highly recommend downloading the ISO to avoid bandwidth issues when using the web installer!

Another gotcha for DirectX developers is that the Windows 8 RTM uses the Intel HD 4000 driver (31/07/12) on the reference Ultrabook, but the drivers misbehave and flicker wildly. I had to dive into the Display driver and replace it with the older Microsoft WDDM 1.2 version (26/05/12) to restore visuals. To me, that’s a driver issue, and I hope I don’t get punished for this by the judges. Sometimes, I’m so cutting edge I chop my nose off!  Here is the fix:

1.       Go to desktop, right click Screen Resolution
2.       Click Advanced Settings
3.       Click Properties button
4.       Click Driver tab
5.       Click Update Driver
6.       Click Browse My Computer for drier software
7.       Click Let me pick from a list of device drivers on my computer
8.       Choose the WDDM 1.2 driver from the list and select Next
9.       Click Close and reset the Ultrabook if promoted

Yet another jungle trap is the 650MB Intel Sensor Solution driver set which lacks certain firmware updates required by the Ultrabook sensors. After some digging around challenge emails, I found a zip called SensorFirmware.zip sent to me by Norman@Intel which contained a file called ‘READ ME with Pictures’ which explained how to update the sensor firmware in six easy steps. Turns out steps two and three don’t play nicely in England unless you delete the <language> section of the dpinst.xml files, and even then step three decided to flip my screen vertically just as I was to press Finish. Tilting the Ultrabook forwards sorted that one out. Step six froze at the end, meaning you had to force as restart of the system but when the Ultrabook rebooted for the final time my sensors where once again working!

One thing to note though is that Windows RTM switches auto-orientation ON by default, which means as soon as you start moving the Ultrabook around, your page starts flipping.  To stop this, go to the Desktop, and right click and select Screen Resolution. Now deselect the ‘Allow the screen to auto-rotate’ tick box, and you are good to go.

Getting Social with the Ultrabook

Another feature of Love Hearts which had been postponed until after IDF was the Facebook, Twitter and Email functionality. On my return, some of the email functionality had started to take shape but the first two had not been touched.  After much research into a good way to involve these social feeds, it turns out the solution was a single command in Freedom-Engine.


Where ‘Hello’ is the link to the message we want to share. By employing the built-in browser, we allow Facebook and Twitter to handle their own log-in and authentication, and instil trust in the user who may not be happy entering their passwords into a third party application.

The WinRT code is a few more lines, but considerably easier than the Win32 equivalent:

Platform:String^ str = ref new String(“www.facebook.com/etc”);
Windows:Foundation:Uri^ uri = ref new Windows:Foundation:Uri(str);
Windows:System:Launcher:LaunchUriAsync(uri);

The last two message styles, Email and App2App rely on PHP scripts sitting on our server, called with a number of HTTP commands built into Freedom-Engine. For all the cleverness contained in the app, a core ingredient of the technology is our server, which runs a series of scripts to take the Love Hearts message text and images, and produces a unique link which can be viewed through a browser, send emails, authenticate and direct queues of messages to the appropriate user for delivery through the app.

In WinRT, the WinSock API is not available for Windows Store apps and instead promotes the use of XMLHttpRequest2 to handle the sending and receiving of HTTP POST and GET calls. These commands alone took two days of solid coding, and a great source of code can be found by searching for the ‘HttpRequest’ sample on MSDN code.

Ultrabook, let there be Sight

In the pursuit of ever more Ultrabook sensors, it is easy to overlook the biggest sensor of them all, the built-in camera. I intended to leave this to the end as it was not an exciting sensor to bedazzle the world with. Nonetheless, it is a vital part of the Ultrabook experience and should not be dismissed out of hand. One of the pearls I took away from IDF is that Perceptual Computing, the technology of understanding what the user is doing when not touching the device, will become key to the evolution of the Ultrabook.

For the here and now, the actual camera capture was catered for rather elegantly by handing it over to the Ultrabooks built-in camera capture tool:

CameraCaptureUI^ dialog = ref new CameraCaptureUI();
dialog->PhotoSettings->CroppedAspectRatio = Size(16, 9);
concurrency::task<StorageFile^> (dialog->CaptureFileAsync(CameraCaptureUIMode::Photo)).then([this] (StorageFile^ file)
{
     m_CameraPhotoFile = file;
}

Coupled with the DirectX texture loader I created earlier in the project, I could use the file to load the image into the app allowing Love Hearts to capture and transmit lovely faces.

I also added WinRT code to allow the user to select a Picture from the Ultrabooks Photo Library, but ran into some issues. The trick here is to copy the image file from the sandbox protected Pictures folder to the Application Data Temporary folder; you can then load and use the image as above. Any developer reading this far into the blog will save about two hours of researching ‘access denied’ and getting nowhere.

Glancing Back

Six blogs is a lot to take in, so before I bid you farewell I would like to list out all the lovely Ultrabook goodness that we've added during our time together:

* A brand new WinRT application created in Visual Studio Express 2012
* An abstraction layer which allows OpenGL to run using DirectX 11 technology
* Using DirectX 3D calls to get super fast 2D rendering on Intel HD 4000
* Used PPL parallel coding to multi-thread loading of shaders and textures
* Re-writing old Win32 functions to their shiny new WinRT equivilant
* Followed Windows 8 guidelines for tile page and app launching graphics
* Tapped the Ultrabook accelerometer sensors to improve app visuals
* Added notification to alter the Windows 8 tile as the app state changes
* Added NFC detection between the Ultrabook and any enabled device
* Adjusted the app to handle any resolution from 1024x768 to 1600x900
* Added multi-point touch detection to scale and rotate on-screen assets
* Used high quality HD graphics to support the highest Ultrabook resolutions
* App checks for new messages every 5 seconds, ideal for Instant Connect
* Added Geo-location commands to detect the city the message is sent from
* One of the presents within the app is a fully working compass
* App detects ambient light and fades to night-mode if the room is dark
* Added Windows 8 in-app purchasing so the app can monetize easily
* Added webcam and photo library support to feed the app with rich media
* Added HTTP commands to access numerous services in the cloud

Phew! I'm sure there's more, but I think that's enough for one post.  I would have loved to add more, such as the Share option, but often in the case of development the act of creating an app produces more app ideas. The mark of a good developer is to know when to draw a line in the sand, bottle those ideas up for another day and share your creation with the world. I'm pretty happy that Love Hearts embodies many of the qualities that make a great Ultrabook app. Something you can pick up and play casually, but also has depth, allowing you create as well as consume. An app that integrates closely with the underlying OS and exploits as many of the unique features of the device as possible.

I also hope I did not get carried away on the technical side and delivered an app that is also bright, colourful, easy and fun to use, provides a touch, sensor and keyboard experience that befits the Ultrabook's capabilities and feels like a natural extension of the Windows 8 user experience.

Farewell

It’s fair to say that developers who want to create a true Windows 8 experience will have to learn WinRT. We’ve learned that the majority of Win32 APIs do not carry over. It’s clear that with the introduction of C++/CX, programmers must adopt a basic grasp of parallel coding to deal with asynchronous operations. Resulting code allows the app to take full advantage of the multiple cores and produce a smoother experiences for the user. The most exciting thing has been the Ultrabook itself, which proved itself most capable as a fully fledged development machine.

It’s been a real privilege to get early access to the reference design Ultrabook, and I’d like to thank the folks at Intel for allowing me to participate in the challenge. I'd also like to wish my fellow challengers the best of luck in their future endeavours, it was an honour to be counted amongst the very best our industry has to offer.

Time for me to perform an ultra-backup, clear my ultra-desk and go to ultra-sleep for an ultra-month. Happy coding and be well.

Lee Bamber
The Game Creators


No comments:

Post a Comment