Battling it out for glory at Battlehack

battlehack-logo

WHAT A WEEKEND! I’m still recovering from the 24 hours of coding and the unhealthy amount of coffee and snacks I’ve ingested during this hackathon.

Two weeks ago, Melbourne was host to the first Battlehack of 2015. “What’s a Battlehack?” I hear you ask. Well Battlehacks are competitions sponsored by Braintree (a Paypal company) where teams of developers face off over 24 hours to create the most creative hack that benefits the local or global community. The venue, which was the Plaza Ballroom on Collins St was well worth a mention. The vast ballroom was decked out with food stations, sleep and massage stations, and most importantly, our hacking stations, or tables of 4 for the long night ahead.

Java 8 – on the path to improving developer sanity

Rewire your brain. Do it now. That’s what Oracle wants you to do with the introduction of the Java 8 SE. Unnecessarily dramatic statements aside, the “JCP” have approved a number of new language features that bring functional programming to Java – and which Oracle are promoting so heavily, they may as well be walking around with “declarative programming is great” inked on their collective foreheads.

Running Geospatial Queries with GeoTools

4331986510_bb69fd7a3c

Geospatial information analysis normally requires pretty complex calculations and transformations between different representation types.  The Google Maps APIs are a great tool because they hide all the complexity of these operations. However when the geospatial information that you need to analyse is not from Google Maps, things get more complicated. Operations like finding polygons representing geographical places, or finding polygons that intercept other polygons, require a lot of time and intricate code to have any acceptable  solution. At Shine we are working on a solution for a big telecommunication customer that requires being able to query large geospatial databases without degrading performance.

Using ‘HugeCollections’ To Manage Big Data

OLYMPUS DIGITAL CAMERA

Introduction

When writing complex software things don’t always go as planned. You implement a new feature that works perfectly well locally and in a test environment, but when your code hits the real world everything falls apart. Sadly, that’s one of the things we all have to deal with as software developers. On a recent project for a major telecommunications client we needed to be able to process more than 20 million records every night. That equated to 5GB of storage and unfortunately the environment where our process was running had up to 4GB of memory.

Processing such a vast amount of data brings a lot of challenges along with it, especially when you also need to combine it with a few more million records that are located in a database. Adding code to retrieve associated information and transform raw data might take an extra few milliseconds per each record. However, when you repeat that operation 20 million times those few milliseconds can easily turn into hours.

So there we were asking ourselves why it was taking so long. Is it an index we forgot to add to the database? A network latency problem? These things can be very hard to pin down.

We needed to think outside the box to get around this one.

Put On Your Streaming Shoes

Saturday-Night-Fever-2

The Kick-Off Meeting

It went something along the lines of:

  • Client: “We have a new requirement for you..”
  • Shiners: “Shoot..”
  • Client: “We’d like you to come up a solution that can insert 2 million rows per hour into a database and be able to deliver real-time analytics and some have animated charts visualising it. And, it should go without saying, that it needs to be scalable so we can ramp up to 100 million per hour.”
  • Shiners: [inaudible]
  • Client: “Sorry, what was that?”
  • Shiners: [inaudible]
  • Client: “You’ll have to speak up guys..”
  • Shiners: “Give us 6 weeks”

We delivered it less than 4.

JavaScript Webapps with Gradle

Duke_the_Ripper IAP4377

Gradle is a build tool on the JVM platform that’s been gaining prominence over the last few years. Gradle’s reach doesn’t stop with JVM languages though.  The most recent releases, 1.10 and 1.11, have improved support for compiling native code (C / C++ / Objective C) and introduced support for the .NET platform. For Android projects, Google have made Gradle the de facto build tool. It’s getting a name for its flexibility, so when the opportunity came to build a single page Javascript webapp we decided to put the latest addition to the polyglot build tool arsenal through its paces.

YOW Conference – Melbourne highlights

YOW Conference 2013

4 Shiners attended YOW Melbourne last week, which is a technology conference held yearly and brings high-profile and savvy presenters to talk on new and current trends in IT.

I’ll start with an overview of the venue, crowd and the sponsors that had stalls in the common area, then dive into a tech report.

Performance Comparison Between Node.js and Java EE For Reading JSON Data from CouchDB

Nodejs

Node.js has impressed me several times with high performance right out of the box. In my last Node.js project it was the same: we beat the given performance targets without having to tweak the application at all. I never really experienced this in Java EE projects. Granted the project was perfectly suited for Node.js: a small project centered around fetching JSON documents from CouchDB. Still I wanted to know: how would Java EE compare to Node.js in this particular case?