Python Tag

Introduction

A website may have hundreds, thousands, or even millions, of public facing pages. When we are responsible for maintaining such a website, it's impractical to traverse it manually looking for broken links. We need an automated testing tool: one which can scan the whole website and log any broken links, so we can get them fixed sooner rather than later. In this blog, I am going to describe a web crawler project which can easily and efficiently achieve the goal. The primary technologies used in this project are Scrapy and Docker.
13 September, 2018 /

Learn more >

pexels-photo-258291.jpeg In part 1, we learnt about recommendation engines in general, and looked at ways to implement a service using the Google Cloud Platform (GCP). In part 2 of the blog series, we are getting our hands dirty on the item-similarity model and TensorFlow implementation of it. This is our first technical blog of the series. Here, I deep dive into the data processing step, the recommendation service, and some hints on how to optimise the code to have real-time responses. You should expect to know how to build a simple item-similarity recommender engine by the end of this blog. So let's get the party started!
24 January, 2018 /

Learn more >

tel-high-res Shine's Technical Excellence Leadership Group (TEL) has had a stellar year! In this post we've pulled together our top picks from 2016 that we think deserve a special shout out before the year comes to a close. But first, a quick recap on what the TEL group actually is. TEL was established in 2011 with the aim of publicising the great technical work that Shine does, and to raise the company's profile as a technical thought-leader through blogs, local meet up talks, and conference presentations. TEL is allocated a yearly budget from the super-duper generous Shine directors, and the members of the TEL group are put in charge of overseeing how it is spent. The budget comprises two parts: money and time. The monetary portion of the budget goes to prizes and bonuses for producing material. The time portion is for staff to draw upon to get away from their day-to-day work commitments and to produce their material. So, now that you know what TEL is all about, let's have a look at the highlight reel from 2016 shall we?
21 December, 2016 /

Learn more >

multiple-seats

My work commute

My commute to and from work on the train is on average 17 minutes. It's the usual uneventful affair, where the majority of people pass the time by surfing their mobile devices, catching a few Zs, or by reading a book. I'm one of those people who like to check in with family & friends on my phone, and see what they have been up to back home in Europe, while I've been snug as a bug in my bed. Stay with me here folks. But aside from getting up to speed with the latest events from back home, I also like to catch up on the latest tech news, and in particular what's been happening in the rapidly evolving cloud area. And this week, one news item in my AppyGeek feed immediately jumped off the screen at me. Google have launched yet another game-changing product into their cloud platform big data suite. It's called Cloud Dataproc.
14 October, 2015 /

Learn more >

licenceToKill A few months ago I joined an exciting project with fellow Shiner Graham Polley, who you might know from such hits as Put on your streaming shoes. This is a follow-up article, discussing the elegant way in which we solved a hideous asynchronous limitation in PHP. My role on the project was DevOps-based, and I was there to build some infrastructure using Amazon Web Services. As the cool kids would put it, I was there to put our client's enterprise application INTO the cloud, or, more succinctly, to build a solution coupling services from two rival cloud service providers and provide a new league of scalability and flexibility. The solution was pretty simple, but, like any simple solution, the little complexities come out along the way, and when you're least expecting them.
19 December, 2014 /

Learn more >