Java 8 – on the path to improving developer sanity

Rewire your brain. Do it now. That’s what Oracle wants you to do with the introduction of the Java 8 SE. Unnecessarily dramatic statements aside, the “JCP” have approved a number of new language features that bring functional programming to Java – and which Oracle are promoting so heavily, they may as well be walking around with “declarative programming is great” inked on their collective foreheads.

Running Geospatial Queries with GeoTools


Geospatial information analysis normally requires pretty complex calculations and transformations between different representation types.  The Google Maps APIs are a great tool because they hide all the complexity of these operations. However when the geospatial information that you need to analyse is not from Google Maps, things get more complicated. Operations like finding polygons representing geographical places, or finding polygons that intercept other polygons, require a lot of time and intricate code to have any acceptable  solution. At Shine we are working on a solution for a big telecommunication customer that requires being able to query large geospatial databases without degrading performance.

Using ‘HugeCollections’ To Manage Big Data



When writing complex software things don’t always go as planned. You implement a new feature that works perfectly well locally and in a test environment, but when your code hits the real world everything falls apart. Sadly, that’s one of the things we all have to deal with as software developers. On a recent project for a major telecommunications client we needed to be able to process more than 20 million records every night. That equated to 5GB of storage and unfortunately the environment where our process was running had up to 4GB of memory.

Processing such a vast amount of data brings a lot of challenges along with it, especially when you also need to combine it with a few more million records that are located in a database. Adding code to retrieve associated information and transform raw data might take an extra few milliseconds per each record. However, when you repeat that operation 20 million times those few milliseconds can easily turn into hours.

So there we were asking ourselves why it was taking so long. Is it an index we forgot to add to the database? A network latency problem? These things can be very hard to pin down.

We needed to think outside the box to get around this one.

Put On Your Streaming Shoes


The Kick-Off Meeting

It went something along the lines of:

  • Client: “We have a new requirement for you..”
  • Shiners: “Shoot..”
  • Client: “We’d like you to come up a solution that can insert 2 million rows per hour into a database and be able to deliver real-time analytics and some have animated charts visualising it. And, it should go without saying, that it needs to be scalable so we can ramp up to 100 million per hour.”
  • Shiners: [inaudible]
  • Client: “Sorry, what was that?”
  • Shiners: [inaudible]
  • Client: “You’ll have to speak up guys..”
  • Shiners: “Give us 6 weeks”

We delivered it less than 4.

JavaScript Webapps with Gradle

Duke_the_Ripper IAP4377

Gradle is a build tool on the JVM platform that’s been gaining prominence over the last few years. Gradle’s reach doesn’t stop with JVM languages though.  The most recent releases, 1.10 and 1.11, have improved support for compiling native code (C / C++ / Objective C) and introduced support for the .NET platform. For Android projects, Google have made Gradle the de facto build tool. It’s getting a name for its flexibility, so when the opportunity came to build a single page Javascript webapp we decided to put the latest addition to the polyglot build tool arsenal through its paces.

YOW Conference – Melbourne highlights

YOW Conference 2013

4 Shiners attended YOW Melbourne last week, which is a technology conference held yearly and brings high-profile and savvy presenters to talk on new and current trends in IT.

I’ll start with an overview of the venue, crowd and the sponsors that had stalls in the common area, then dive into a tech report.

Performance Comparison Between Node.js and Java EE For Reading JSON Data from CouchDB


Node.js has impressed me several times with high performance right out of the box. In my last Node.js project it was the same: we beat the given performance targets without having to tweak the application at all. I never really experienced this in Java EE projects. Granted the project was perfectly suited for Node.js: a small project centered around fetching JSON documents from CouchDB. Still I wanted to know: how would Java EE compare to Node.js in this particular case? 

JavaOne Day 3

With the previous two days of JavaOne 2013 now past us, it was time to get a little more practical and look more closely at some concepts introduced earlier in the week.  Today wasn’t going to run as late into the evening as previous days since Oracle had their appreciation event beginning at 7pm.  Personally, I was looking forward to the shorter day, getting a break from ‘conference brain’ and getting some time to consolidate what I had already picked up this week.