AI Driven Retrospectives

•December 3, 2017 • 1 Comment

With the drive of AI and chatbot recently, its of no surprise to find ScrumBot.  What I think would however be more interesting is to hook the bot into a wider array of data outside of the Slack/Hipchat/Microsoft Teams world, maybe levering product metrics, as well as SDLC metrics.  Standuply appears to hint at this via the third party metrics integration.  I suspect if the bot could leverage a appropriate metrics, its would be powerful not only from a retrospective, but also from a pair engineering experience as well


Agile influences Behavior Change

•November 15, 2017 • Leave a Comment

There is often debate around what Extreme Programming (XP) / agile practices work, and what doesn’t – hotly debate are often pair programming and test-driven development.  What is often missed is that agile has an underpinning, which is the drive for behaviour changes to deliver improvements (quality) in the deliver of software.  This may seem obvious, but its worth calling out explicitly – Connection and here.

Whichever agile techniques you find that work for you and the team, just remember, the need for behaviour change is an ongoing process to drive quality improvements, its not that adoption of any particular agile technique itself will improve quality.


Recurrent Neural Networks (RNNs)

•August 20, 2017 • 1 Comment

Some while ago a data scientist colleague pointed me at “Practical Machine Learning With Event Streaming” over on the Monzo blog.  The key takeaway if you don’t have time to read the article fully is:

RNNs specialise in making predictions based on a sequence of actions (known as an “event time series”), e.g. user logs in → taps on “money transfer” → encounters an error.

After reading the article, I got to wondering if you can take the event stream from the Software Development Life Cycle (SDLC) process and use RNNs to predict defects?

Bots As A Mean To Motivate Behavior Change

•June 5, 2017 • Leave a Comment

Often the underlying message around Extreme Programming (XP) and agile in delivering software is actually a behaviour change required by the person/team.  Behaviour change is hard, we as individual are often stuck in our ways 🙂  Hence I was interested to come across “Chatbots As A Mean To Motivate Behavior Change”, especially given the last few years buzz around AI and bots 🙂

“Pushy AI Bots Nudge Humans to Change Behavior” also provide some thought “Researchers use artificially intelligent bot programs to stimulate collaboration and make people more effective”

Could a chat bot, driven by a neutral network or appropriate algorithm, steer a software engineer towards improving the codification of application changes ( story/requirement)?

Update: Few more recent articles provide food for thought on the above:

Evidence of Success

•April 11, 2017 • Leave a Comment

A few thoughts if you are about to venture down the road of building a software applications:

  • How will you know when you are DONE on implementing the requirements (Stories)?
  • How will you validate to the product owner that you are DONE?  Maybe consider traceability and linkage of stories to acceptance criteria to acceptance tests?
  • Ticking a box to say you have implemented all the requirements is in many ways immediate stale as soon as the code is “enhanced” post the box ticking exercise.
  • When in production, how will you know if the application is “broken”?  Considering solutions for logging, monitoring, availability etc
  • Logging is only as good as the data logged 🙂  Over use of logging will mean there is the wrong noise to signal ratio 🙂  Likewise to monitoring
  • If the application is “highly available”, prove it, in production 🙂

You could think of this as Test Driven Developer->Behaviour Driven Developer->Evidence Driven Developer, or even Evidence Driven Engineering

Know Your Stack – Neo4j

•February 22, 2017 • 2 Comments

Sometimes things go south in production.  Often its a combination of events that have generated a perfect storm.  The storm being production outages that are difficult to diagnose due to lack of understanding of the application stack, tools, and knowledge.

When attempting to diagnose an issue, its important to piece together the situation though evidence – log files, monitoring tools, etc.  Your own codebase can often be improved with appropriate logging and inclusion of monitoring tools/UI’s.  Issues often come about with off-the-shelf products, and lack of knowledge around how to understand the inner workings.  One example in Neo4j.  What follows is a few hopefully helpful suggestion to understand what is going on in your application stack:

  • Neo4j has a query.log which is extremely useful in tailing to view the queries being run, and the time to execute each query.
  • Neo4j browser (port 7474 by default) has a “:play sysinfo” which has some basic information that can prove useful e.g the Transaction tile, and in particular “Current”.  If Current remains move than 0 for a duration, when you expect it to be 0, you could have long running transactions 😦
  • Neo4j 3.1+ has improved query management.  In particular its now possible to see what queries are running, and kill queries if required
  • Reading documentation before using a product is key.  Remember the Open Files for Neo4j.

Data Engineers

•February 7, 2017 • 1 Comment

The refinement of roles and titles in the ‘Big Data’ world continues to evolve.  Previously I blogged about the many roles in the data pipeline engineering world, now Maxime Beauchemin adds to the context with an article on “The Rise of the Data Engineer”.

In many scenarios I think the ETL concepts of old are dead.  As I’ve blogged about previous, its more a ELT pipeline, since all raw data is worthy of keeping.

I’d say the below comment is particularly key, especially if you’ve got an efficient Continuous Delivery pipeline:

Just like software engineers, data engineers should be constantly looking to automate their workloads and building abstraction that allow them to climb the complexity ladder.