Author: Shine Solutions Group

Shine’s TEL group was established in 2011 with the aim of publicising the great technical work that Shine does, and to raise the company’s profile as a technical thought-leader through blogs, local meet up talks, and conference presentations. Each month, the TEL group gather up all the awesome things that Shine folk have been getting up to in and around the community. Here’s the latest roundup from what’s been happening.
Shine’s TEL group was established in 2011 with the aim of publicising the great technical work that Shine does, and to raise the company’s profile as a technical thought-leader through blogs, local meet up talks, and conference presentations. Each month, the TEL group gather up all the awesome things that Shine folk have been getting up to in and around the community. Here’s the latest roundup from what’s been happening.
I have been using BigQuery for over 2 years now at Shine. I've found it to be a great tool that is both incredibly fast and able to handle some of our largest workloads. We are processing terabytes of data per day, and each day an extra billion records are added to the store. But unfortunately this growth is also increasing our costs of running queries. While BigQuery is extremely fast and parallel, it comes at the cost of needing to scan and pay for every record of the columns you are querying. Without the indexes offered by conventional databases, a full table scan is needed for each query. Not only that but when you query large amounts of data the speed of your query slows down:In this post I'll talk about how we used table partitions to increase the performance of our queries and avoid query slowdowns.

"What the Fudge?"

I use Google BigQuery a lot. On a daily basis I run dozens of queries, use it to build massively scalable data pipelines for our clients, and regularly help new users navigating it for the first time. Suffice it to say I'm somewhat accustomed to its little quirks. Unfortunately, the same can't be said for the new users who are commonly left scratching their heads, and shouting "What the fudge!?" at their monitors. Here's the top three WTFs that I regularly hear from new BigQuery users:

Will this post interest me?

If you use (or intend to use) Google Cloud Dataflow, you've heard about Apache Beam, or if you're simply bored in work today and looking to waste some time, then yes, please do read on. This short post will cover why our team finally took the plunge to start porting some of Dataflow applications (using the 1.x Java SDKs) to the new Apache Beam model (2.x Java SDK). Spoiler - it has something to do with this. It will also highlight the biggest changes we needed to make when making the switch (pretty much just fix some compile errors).
  Shine’s TEL group was established in 2011 with the aim of publicising the great technical work that Shine does, and to raise the company’s profile as a technical thought-leader through blogs, local meet up talks, and conference presentations. Each month, the TEL group gather up all the awesome things that Shine folk have been getting up to in and around the community. Here’s the latest roundup from what’s been happening.

5 tips on form design to improve your relationship with users

Filling in a form online is one of the most important points of interaction a user has with an organisation. And we interact with them often. We fill in tax forms, grant applications, make online purchases or sign up to dating sites. Forms can be the first step in a relationship with an organisation, or the final step in a journey to achieve a goal. For example get a grant, a drivers license or a partner in crime. Sometimes not filling them properly can carry unpleasant consequences like an interrogation by immigration officers at the airport, or your profile on OkCupid matching you with the wrong date.💔
“A form [ ] collects information from at least one party, and delivers it to at least one other party, so a product or service can be provided.”~Jessica Enders
The role of a UX designer is to help create easy, fast and productive form experiences. To entice users to fill in forms. As form design expert Jessica Enders states, designers should “create an optimal user experience, such that the needs of both the users and the owner of the form [organisation that owns the form] are met.”  
Shine's TEL group was established in 2011 with the aim of publicising the great technical work that Shine does, and to raise the company’s profile as a technical thought-leader through blogs, local meet up talks, and conference presentations. Each month, the TEL group gather up all the awesome things that Shine folk have been getting up to in and around the community. Here’s the latest roundup from what's been happening.