IaaS

Cloudflare Dev Workshop 2020 In mid-February, I had the privilege to attend the first Melbourne Cloudflare dev event. This was just one of a series of sessions they ran across the country to reach out to developers and help educate people around their thinking and the...

04 March, 2020 /

Learn more >

No food reviews here I'm afraid

This year I was incredibly lucky to score a coveted ticket to YOW! in beautiful Melbourne. I was also asked to be a track host for a couple of sessions, so that was quite an honour too. This post is a whirlwind wrap-up of the conference, and only includes my favourite talks from the two day event. If you're hoping to hear detailed reviews on how the coffee/food/WiFi/venue was, then you'll be greatly disappointed (it was all great BTW).
01 December, 2017 /

Learn more >

Will this post interest me?

If you use (or intend to use) Google Cloud Dataflow, you've heard about Apache Beam, or if you're simply bored in work today and looking to waste some time, then yes, please do read on. This short post will cover why our team finally took the plunge to start porting some of Dataflow applications (using the 1.x Java SDKs) to the new Apache Beam model (2.x Java SDK). Spoiler - it has something to do with this. It will also highlight the biggest changes we needed to make when making the switch (pretty much just fix some compile errors).
22 May, 2017 /

Learn more >

The decline of Stack Overflow?

A few months back I read this post from 2015 (yes, I know I'm a little late to the party) about how Stack Overflow (SO) was in serious decline, and heading for total and utter oblivion.  In the post, the first item to be called  out was that SO "hated new users":
Stack Overflow has always been a better-than-average resource for finding answers to programming questions. In particular, I have found a number of helpful answers to really obscure questions on the site, many of which helped me get past a road block either at work or in my hobby programming. As such, I decided I’d join the site to see if I could help out. Never before has a website given me a worse first impression.
At the time, I remember thinking that this seemed like somewhat of an unfair statement. That was mostly down to the fact that when I joined the community (many years ago), I had fond memories of a smooth on-boarding, and never experienced any snarky remarks on my initial questions. Yes, gaining traction for noobs is very, very hard, but there is a good reason why it exists. For me, SO is invaluable. How else would I be able to pretend to know what I'm doing? How else could I copy and paste code from some other person who's obviously a lot smarter than me, and take all the credit for it? Anyway, once I had read the post, and gotten on with my life (e.g. copying and pasting more code from SO), I did't think too much more about the post. Maybe I had just been lucky with my foray into the SO community? However, just last week, I was reminded of that post once again, when I noticed that BigQuery (BQ) now has a public dataset which includes all the data from SO - including user comments and answers. Do you see where I am going with this yet? If not, then don't worry. Neither did I when I started writing this.
20 December, 2016 /

Learn more >

*Updated on 16th December 2016 - see below With the announcement of Amazon Athena at this year's AWS re-invent conference, I couldn't help but notice its striking similarity with another rival cloud offering. I'm talking about Google's BigQuery. Athena is a managed service allowing customers to query objects stored in an S3 bucket. Unlike other AWS offerings like Redshift, you only need to pay for the queries you run. There is no need to manage or pay for infrastructure that you may not be using all the time. All you need to do is define your table schema and reference your files in S3. This works in a similar way to BigQuery's federated sources which reference files in Google Cloud Storage. Given this, I thought it would be interesting to compare the two platforms to see how they stack up against each other. I wanted to find out which one is the fastest, which one is more feature rich and which is the most reliable.
09 December, 2016 /

Learn more >

At Shine we're big fans of Google BigQuery, which is their flagship big data processing SaaS. Load in your data of any size, write some SQL, and smash through datasets in mere seconds. We love it. It's the one true zero-ops model that we're aware of for grinding through big data without the headache of worrying about any infrastructure. It also scales to petabytes. Although we've only got terabytes, but you've got to start somewhere right? If you haven't yet been introduced to the wonderful world of BigQuery, then I suggest you take some time right after this reading this post to go and check it out. Your first 1TB is free anyway. Bargain! Anyway, back to the point of this post. There have been a lot of updates to BigQuery in recent months, both internally and via features, and I wanted to capture them all in a concise blog post. I won't go into great detail on each of them, but rather give a quick summary of each, which will hopefully give readers a good overview of what's been happening with the big Q lately. I've pulled together a lot of this stuff from various Google blog posts, videos, and announcements at GCP Next 2016 etc.
19 August, 2016 /

Learn more >

Databases are the backbone of most modern web applications and their performance plays a major role in user experience. Faster response times - even by a fraction of a second - can be the major deciding factor for most users to choose one option over another. Therefore, it is important to take response rate into consideration whilst designing your databases in order to provide the best possible performance. In this article, I’m going to discuss how to optimise DynamoDB database performance by using partitions.
27 June, 2016 /

Learn more >

Show more
Loading...