Beam me up Google – porting your Dataflow applications to 2.x

Will this post interest me?

If you use (or intend to use) Google Cloud Dataflow, you’ve heard about Apache Beam, or if you’re simply bored in work today and looking to waste some time, then yes, please do read on. This short post will cover why our team finally took the plunge to start porting some of Dataflow applications (using the 1.x Java SDKs) to the new Apache Beam model (2.x Java SDK). Spoiler – it has something to do with this. It will also highlight the biggest changes we needed to make when making the switch (pretty much just fix some compile errors).

Triggering Dataflow Pipelines With Cloud Functions

Do you have an unreasonable fear of cronjobs? Find spinning up VMs to be a colossal waste of your towering intellect? Does the thought of checking a folder regularly for updates fill you with an apoplectic rage? If so, you should probably get some help. Maybe find another line of work.

In the meantime, here’s one way to ease your regular file processing anxieties. With just one application of Google Cloud Functions, eased gently up your Dataflow Pipeline, you can find lasting relief from troublesome cronjobs.

Highly concurrent applications with Java and Akka

There are applications that execute such complex tasks that if they didn’t use a concurrent processing model, they would be so slow as to be unusable. This group of applications includes data analytics, real-time games and recommendation systems.

Even with modern programming languages that support concurrency, we are faced with the task of coordinating multiple threads, handling synchronisation and the constant possibility of race conditions. These make it difficult to write, test and maintain code, discouraging many developers from implementing better and faster solutions for their problems.

multithreading

In this blog entry we are going to take a quick look at the Akka toolkit, its main concepts and some code examples in Java. For further information about this topic, please check the official documentation at http://akka.io/

YOW! Connected 2016: The typed-functional future of UI platforms

I just spent a couple of days at the YOW! Connected conference and had a great time, despite nursing a bit of a cold. There were a tonne of great talks at the conference covering a wide range of topics, but in this post I’m going to briefly reflect on one specific trend that interested me at the event: the way in which UI platforms are advancing to adopt modern languages, and are even influencing each other in the process. The end-result: they’re all moving towards languages that are both functional and statically typed.

Full disclosure: This year I was a member of the programme committee for the conference. So in writing this post, there’s a bit of a risk that I’m creating an echo chamber for myself. All I can really say in my defence is that I hadn’t consciously made these connections in advance – it was only afterwards that I saw a trend!

Shiner to present at very first YOW!Data conference

 

Shine’s very own Pablo Caif will be rocking the stage at the very first YOW! Data conference in Sydney. The conference will be running over two days (22-23 Sep) and is focused big data, analytics, and machine learning. Pablo will give his presentation on Google BigQuery, along with a killer demo of it in action. You can find more details of his talk here.

Shine’s Pablo Caif to present at GCP Next 2016!

next

Shine is extremely proud to announce that Pablo Caif has been invited to present at GCP Next 2016, which is Google’s largest annual cloud platform event held in San Francisco.

Pablo will be presenting on the work Shine have done for Telstra, which involves building solutions on GCP to manage and analyse their massive datasets. More specifically, the talk will focus around Google’s two core big data products – BigQuery & Cloud Dataflow.

Pablo will be presenting on Thursday 24th March in the ‘Data & Analytics’ track. Be sure to pop by and say “g’day” if you are going to the event! You can find more information about GCP Next 2016 here.

 

Securing your Spring App using 2FA

2a_02 Not so long ago, a good old username and password were considered more than enough to secure access to our applications and favourite web sites. But back then, nobody could have imagined the countless ways in which a hacker can now get a hold of our precious login credentials. From software exploits to social engineering, security has been drawn into the spotlight like never before, and software developers must really think hard about security when building any type of software solution. In this blog post, I’ll explain how you can secure your Spring applications using 2FA (Two Factor Authentication).

Spring Data REST and Projections

Project_Data

Introduction

In recent years, Spring has become much more than just a dependancy injection container and an MVC web application framework. Nowadays, it’s the go-to for building enterprise solutions due to the fact it has a fantastic community built up around it, and it has a multitude of projects that makes every developer’s life that little bit easier! In this blog post, I’m going to briefly introduce Spring Data REST, and how we used it and an unknown feature called ‘projectionson a recent project.

Test Driving Google Cloud Dataflow

Bq_tOGxCMAELB4k

Back in June 2014, at the annual Google IO in San Francisco, Google unveiled their newest, and much hyped cloud product, Cloud Dataflow. The demo they did that day, using a live twitter feed to analyze supporter sentiment during the 2014 world cup, got my mouth watering at the prospect of working with it. It looked downright freaking awesome, and I just couldn’t wait to get my hands on it to take it for a spin.