The Art of The Bodge

Leave a reply

As tech folk, we seem to obsess over best practices, beautiful & maintainable solutions, and using the newest technologies in smart and elegant ways.

But what of the humble “JFDI” bodge? When everything is on the line – when something has failed in production, critical functionality isn’t working and we can’t rollback, or we need to urgently build a tool or run some hacky solution to resolve some regulatory data issue fast – I will say it, I love a good bodge job.

There is something about the context – the high stakes, the pressure, the business importance, complications and genuine impact of it all – of scrambling to chuck together a dirty console app written with hopes and dreams, or getting out the virtual duct tape in some other way, and ultimately saving the day. It’s exciting! And often the most creative solutions come from these moments, where requirements don’t exist aside from “fix it”. I’m unashamed to say it – dirty bodge jobs are unabashedly my favourite part of being a software developer. These are some of the situations that make you feel alive and like you’re using some creative ingenuity to find the best (not “ideal”, but quickest and most effective for the issue) resolution, and having some real impact, rather than just churning out code in sprint after sprint monotony. 

I think I might be in the minority here – but I also think my skills lie, generally, not like many developers who I admire (i.e. I don’t consider myself a super skilled writer of code), but more in being able to find the balance between what is the “best” engineering solution vs what is the best solution for the context. 

And ultimately, I find I get more satisfaction, these days, from something I did having a good real world impact, rather than the work itself being a satisfying piece of engineering. I’m not knocking the positives of those who prefer the latter, by any stretch of the imagination – it’s just not what personally gets me out of bed in the morning. 

Yes, with perfect solutions and perfect code, we wouldn’t need them, and businesses would be better off without these stressful, scary situations – but while that is realistically unattainable and not all factors and edge cases can ever be fully considered… I am pretty happy to get to do a bodge job!

The Battle Against Imposter Syndrome

Leave a reply

Well, it’s been a while since I posted here, and this will (unusually) be quite a personal post. But for (hopefully) good reason – I hope that it’s one that could benefit even just one person reading this. Not to mention, I, increasingly, have realised the importance of being open – not only for ourselves, but for the effect that it has on others. That said (and perhaps this is evidence that I am still fighting this battle), I feel the need to preface and say that I hope this is read in the spirit of what I intend as being honest for the benefit of somebody reading who may be suffering in silence, and not in a self-pitying or, conversely, self-aggrandising way, when I share some personal ups and downs in this post.

The last few years have been pretty turbulent for me. I’m certainly not alone in this; it’s an understatement to say that we’ve had a difficult time globally, through a pandemic, rising mental health crises, cost of living skyrocketing, political divide and war (and many more), we have plenty of fairly universal truths that say that the world hasn’t been so kind to many people recently. On top of this, of course we all have our own personal factors that could push this difficulty further.

I’m the type to downplay my own problems for want of not appearing like I’m overreacting, oversensitive, or unable to handle pressure, but after a series of events starting a couple of years (or so) ago, I was not always able to hide that. And now on reflection, I can say it honestly – I did have an abnormally bad time for a while and it affected me.

Without getting too deep into the specific personal details of each situation, within just over a year, I changed job, had two surgeries with longer-than-anticipated recovery (being some element of physically inactive for around a year), went through a long breakup, had some financial worries, family health scares, supported friends with some exceptional life/mental health circumstances of their own, and changed job again. Most of the time, I acted as if these events were “just one of those things” and I was rolling with the punches, but there’s only so much you can just override, and in hindsight I was often living in a “fight or flight” mindset. 

Why am I mentioning all this? It’s not just for my own catharsis; but because I know I can’t be alone in one particular element. According to this blog post by Subhasish Dutta on Turing, a survey of developers found that 58% said they suffered with Imposter Syndrome; and I know first-hand how external factors can only make that worse. It impacts your self-image, the way you think, your ability to focus, the decisions you make, and importantly – your enjoyment of life and the things you should be proud of. You believe you “can’t” even though you “do”.

So here’s a story to see if this resonates with anyone reading: I’ve always been the type to feel the need to prove myself, and internally doubt my own abilities. It didn’t matter what I achieved – there was some reason why it didn’t “count”, or someone else was better, or I wouldn’t be able to do it in X or Y environment, or I’m just fooling everyone; I achieved said “thing” because I played the system, took some shortcut, convinced everyone I’m capable of something that I’m really not. There’s always a reason other than “I’m actually good at something”. The further I got in my life, the more things I accomplished, the more I felt like this imposter; and having so many events bring me down a peg or two only compounded this when I previously could suppress it. If I was having a hard time concentrating or solving a problem, my first thought was because I must be a fraud, and not because of the myriad of things that could be impacting me. We are human and it’s natural.

I’m not going to totally absolve my own responsibility for my feelings and behaviour; of course I have weaknesses, I also could’ve looked after myself better, and there are things I chose to do prior in life that if I’d done differently, I might be in a better position. But nobody is perfect, life’s journey is not a straight road, and I also really believe it’s so important to be able to recognise the objective truths of situations, to be able to conquer this imposter feeling – and that includes not only seeing the good things you’ve done for what they are, but if something was negative or affecting you, not to entirely brush it off or think “I should’ve just done better”.

So in that spirit, and for want of ending this on a more positive note, I am going to finish by highlighting some of my objective positives from the last few years, despite the setbacks. This is a real challenge for me – I am terrible at selling myself. But I think that until we so-called imposters can do this and not worry that we’re coming across as bragging, or that in the back of our minds, these things don’t count – we will remain feeling unsatisfied by all of the things we should take pride in:

  • In my spare time, I built a web app that benefited my local community, and was used by over 10% of the voting public in my country to help make informed decisions on who to vote for
  • I recovered from my operations, lost weight, and am now fitter than I have been for years
  • I have been a supportive, loyal friend and family member to people close to me through hardships, even when I had my own struggles
  • I did have success in my job despite being at the peak of these events – and while I might’ve done some things differently, I was still able to keep going with life, keep producing good work, and largely not let it impact the reason that I was being paid
  • I took a step up to become a Senior Developer, and have accepted that this title doesn’t mean I have to be an oracle of all knowledge, but rather that my experience and some of the qualities that I have (which are unique to everybody) allow me to first and foremost be able to support younger devs, help them grow, and enable them to resolve issues that they’re struggling with – which is massively rewarding. And I am here on merit, not from 10 years of fooling everyone!
  • Despite some financial worries, I found ways to make some need for seeing more of the world a reality, and continue to do so
  • I achieved some personal dreams during this time, however small in comparison to others, they were milestones to me – e.g. yeah, I didn’t headline Glastonbury or Hellfest, but I did perform a dream gig with likeminded musicians to a sell out crowd in a small venue where I live!

It’s so important to remember context and not compare yourself to others. There will always be someone “better” than you. There is always room for improvement. But can you be content in what you’ve done? Of course – and that should be the bare minimum. 

We mustn’t lose sight of the things we’ve done along the way. In my case, at the time, just landing a developer job back in 2014 was an enormous achievement for me. I’d come from a family where nobody had ever been to university and we couldn’t afford to send me, but I spent 3 years working jobs that I didn’t enjoy to fund myself before I went to study Computing, received great grades, and ultimately achieved the fruit of that journey, and it changed my life. I’m sure we all have things we can and should be immensely proud of that we mentally discard in favour of the internal bully, and I just sincerely hope that it’s something that someone reading this can be mindful of in their own life. Otherwise, we’re just living for others, being our own worst enemy, and never feeling the confidence and satisfaction that should come with being the person you had once strived to be.

An easy way to drastically improve JsonSerializer performance (System.Text.Json)

Leave a reply

A .NET 6 project that I’ve been contributing to uses the new(ish) System.Text.Json JsonSerializer to serialise/deserialise data prior to storing to, and after retrieving from, a distributed cache.

I started to notice some performance degradation as I increased our use of caching in this application (and therefore more JSON serialising/deserialising going on), so set to work on investigating why this was the case.

To note – we had been retrieving the JsonSerializerOptions object from a static method rather than using a static variable, an approach even used by some tutorials, e.g:

    private static JsonSerializerOptions GetJsonSerializerOptions()
    {
        return new JsonSerializerOptions
        {
            PropertyNamingPolicy = null,
            WriteIndented = true,
            AllowTrailingCommas = true,
            DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
        };
    }

And one of our usages of this in the cache context (we were also deserialising in the same way when getting from the cache):

 private static Task SetAsync<T>(this IDistributedCache cache, string key, T value, DistributedCacheEntryOptions options)
    {
        var bytes = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(value, GetJsonSerializerOptions()));
        return cache.SetAsync(key, bytes, options);
    }

Testing – Huge Performance Difference

I’ve written a couple of benchmark tests comparing the use of a “GetJsonSerializerOptions” method to return an JsonSerializerOptions instance on de/serialise vs using a static variable. The code and data used for both tests is identical, aside from the approach with the options instance – it loops through and both serialises and deserialises an object 10,000 times, and logs the time taken and memory allocated:

The test – 10k serialises + deserialises of a dummy “CustomData” object:

Static Method: 45,850.30ms mean time, 1397.81MB Memory Allocated

Static variable: 62.42ms mean time, 32.65MB Memory Allocated

Yep – over this 10k loop, the static method approach allocated over 1.3GB more memory, and took roughly 734.5x more time, vs using a static variable…! I then double checked the same approach with dotTrace – bear in mind there is a margin for error and deviation…


~44.7seconds for the Static Method, 80ms for the Static Variable.

At the bottom of the post is a gist to show that both tests are identical (10k loops of serialising and deserialising the same data, with the only difference being the JsonSerializerOptions).

Why?

This behaviour is due to the way the JsonSerializer fundamentally works. When serialising/deserialising an object, the JsonSerializer has to perform a “warm-up phase“, which involves generating and caching metadata related to how to de/serialise the type. When the JsonSerializerOptions are re-used, it can simply use the same metadata that it previously cached for the next time an object of that type is processed – however, with a method returning a new JsonSerializerOptions instance every time, it cannot use the cached de/serialising metadata for a type that has already been processed, and has to generate this anew every single time it serialises or deserialises any data. This is an expensive process, and results in much slower execution + a huge amount more memory being allocated. If part of your system relies on JsonSerializer before storing or after retrieving data – this could result in slow performance, and potentially memory leaks and timeouts.

Solution

The change is very simple. Replace (and change any references to) any method/anywhere inline where you might be creating a new JsonSerializerOptions object repeatedly, with a static variable, such as:

private static JsonSerializerOptions _serializerOptions = new JsonSerializerOptions
    {
        PropertyNamingPolicy = null,
        WriteIndented = true,
        AllowTrailingCommas = true,
        DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
    };

Gist

First World Problems: Parking, Security & User Experience Thoughts/Rant

Leave a reply

I live in an area where car parking is scarce, and as such, myself and my girlfriend rent a parking space in a small, unsecured car park nearby. We pay about £150/month for the privilege, and are currently able to swap which car is in our space as and when we like. In an ideal world, we’d share a car; but for us, this isn’t an ideal solution – our schedules sometimes conflict, and there are times where we both need our cars, or when having a specific car closer to the house is beneficial (I’m a guitarist, so if I may need to be temporarily close to the house to load up my car for a gig, for example).

With this system, we’re also afforded the ability to find rare on-street parking in advance of having a visitor with reduced mobility, so we can at least make sure our visitors can park near our house!

Recently, our letting agent installed a CAME parking/”boom” barrier at the gate of our car park, without contacting us. On a piece of A4 paper attached to the barrier was a notice in small text, advising that this barrier would be active from 1st May – I use my car fairly regularly, and first noticed this in early April, so not a huge amount of notice. The paper went on to advise that we must visit the letting agent’s office to retrieve a key fob for the gate before this date.

Problem 1 – Lack of contact, timescale – ignorance

We are still currently in the throes of a pandemic, despite an increasing amount of the world now acting as if we are not. If I, as a young, fairly observant, healthy person, who uses their car regularly, only noticed this change by chance within 2-3 weeks until the date required, imagine the following user stories:

1. A (potentially elderly) person who uses their car sparingly

2. Someone who stores a classic car or motorcycle in one of the garages within the car park, only using it when the weather and their schedule suits.

3. Someone who is off island for a relatively long holiday, bearing in mind that travel hasn’t been possible for many for over two years, and we have had a decent run of bank holidays that would make this an attractive idea at this time of year.

4. Someone who has contracted COVID-19 and has to self isolate for ~two weeks.

With any of these use cases, due to not being contacted directly, the user would likely arrive to use their vehicle past the “cut-off date” to find that they cannot leave or enter the car park, their vehicle effectively trapped.

This is the equivalent of forcing two-factor authorisation to login to your website, without directly notifying existing users of the change – assuming that they would attempt to login within a two-week window of making this change AND notice the small print on their account page, AND be in a position to accommodate this fairly immediately.

Problem 2 – Security being implemented without consideration of impact

I am all for increased security, particularly in software. But developers see this kind of half-baked thinking all the time – security being bulldozed into a solution/platform without consideration for the impact on all facets of the business, use cases, or essential work. A security solution should be put in place in a way that has clearly considered these aspects, and the impacts of such a change. Want to introduce a restricted “jump server” so that engineers cannot directly connect to production systems? Sure, makes sense from a cybersecurity sense! We don’t want anyone who has been able to access an internal account to directly connect to production, or rogue internal actors to steal sensitive data (or, we want to make sure such actions are difficult/auditable). Totally get that – so you’d probably want to restrict access to the internet, along with ability to copy data to and from the jump server. Fine – get it implemented. Sounds great – until something major goes wrong in production, and a developer needs to urgently run some code against the live system to fix it – in an ideal world, this wouldn’t happen, but this is a pretty big crisis and solutions need to be quick. How would they get the code onto the jump server without the ability to copy data? Perhaps they could retrieve it from source control – until you realise that internet access has also been locked down. You must consider these cases and have solutions in place that toe the line between security and usability/agility, before the security solution is chucked in – otherwise, users will just find hacky, unsecure ways around your security, defeating the purpose – the alternative is that users will be unable to do their job, potentially with a greater financial detriment than the security measures would prevent.

I’m certainly not saying that security should be ignored, or backdoors implemented to make certain people’s lives easier – but a balance must be struck and at the least, well understood and thought-out processes must be in place before security implementation (that allow a team to work at least similarly quickly to before restrictions are implemented).

Anyway – onto the parking barrier! As I mentioned earlier, my use case involves using the parking space for multiple vehicles, as I believe is the case for other car park users. This is at best a mild inconvenience, but it’s a good example of a real-world haphazard security implementation: we are only given one key fob per parking space! It’s not always easy/possible to hand-over the fob prior to another user using the space – if I am currently using the space, and have a limited timeframe to move my car before my girlfriend needs to use the space, I would usually just need to find a road parking space nearby, and the space is all hers. Not so now – I now need to factor in a hand over of the fob between myself moving my car, and her arriving in hers – or wait around the car park to open it for her. This isn’t always possible due to the conflicting schedules mentioned above, or what if the person holding the fob is currently far away when the space is needed – sure, I could drive to her and hand her the fob then, but that’s just not always possible! Perhaps she could pick it up from the house – but that would rely on one or both of us being able to find a space close enough to not be an inconvenience, and for situations like the current fob-holder not needing to drive elsewhere and requiring the space when they return. There are solutions, but they are a little clunky compared to the previous use we enjoyed for £150/month.

These fobs cost approximately £5-10, and require a couple of minutes of pressing a button to pair with the gate – so I assumed that our letting agent may have considered this eventuality. Surely they can just provide us with a spare fob?

Nope! One fob, one parking space. If we want a spare, or lose our current fob, we are held to ransom to the tune of an additional one month rental (so, £150 for a £5 fob and two minutes of pairing). They have clearly not considered all use cases and are either blatantly overcharging people, or have bought the bare minimum of fobs, thus requiring some contractor to overcharge them for the pairing of a new one.

Like any mischievous developer would in a situation like this, my first thought was: how can I overcome this annoyance? I’m getting a worse, less convenient service for the same price, and I find it frankly insulting to be told a cheaply made, tiny circuit board + two minutes of labour will cost me the same as one month of in-demand parking, just to achieve a similar freedom to that which we once enjoyed. I opened up the fob, checked the board online to see if it can be easily cloned – hoping the fob was a “fixed code” (i.e. easily cloneable) so I could get my own copies for a couple of quid. Sigh… It’s a rolling code system, based on the HCS301 microchip. More secure, but pretty unnecessary for a parking barrier in this use case (where someone malicious could simply walk around it) in a car park with about 30 spaces. I find it hard to believe anyone would clone the fixed code to use a parking space without paying for it, given that the main thing they are trying to stop is opportunistic drivers spotting an empty space there among the lack of on-street parking… So, these systems are technically cloneable, but the cloned code would only work once (as a new code is generated with each use, using a rolling code), or, if you were able to clone the entire chip/serial to produce basically an identical copy of the code sequence, one fob would always be out of sequence with the other, requiring multiple button presses before the other fob was actually “in sync” with the code that the barrier was expecting. If the letting agent had used a fixed code, it would be less secure, but the deterrent of a barrier would be there for presumably the main reason that it exists (to stop opportunistic parkers who can’t find street parking in the area from using these spaces) – and the letting agent themselves could produce extra fobs easily, cheaply, and without anyone needing to physically visit the barrier.

This has been quite a long rant for such a minor inconvenience, but I think it illustrates a pretty big gripe of mine in the software dev world, with a real-world example – the best security solution must consider context, risk level of the thing you are protecting, use cases, and have plans in place for users to complete the same actions as before (provided they are allowed) – BEFORE implementation!

First post!

Leave a reply

Well, I’ve finally decided to start a techy blog! Just a little intro on me, I’m a software developer who has been tinkering with tech since, well, as long as I can remember – I’m now thirty, and I’ve found myself somewhat lackadaisically uninspired with tech stuff for a little while (for the first time in my life) – partly due to real life things, and partly due to finding myself in a bit of a rut. Now that the dust has settled a bit on the RL side of things, I’ve found some inspiration recently after a few conversations with various tech folk who are much smarter than I, and it’s been the kick I needed to get back into what I love doing.

So, I’m going to be documenting some of the things I’m up to, whether that be fun little projects like building an arcade machine, personal stuff like my guitar/music/video gaming interests, my general thoughts and ramblings, or, most importantly to me – my various software dev projects, particularly while I try to modernise my skills a touch.

I fully expect that this blog will likely not be read by many, but it’s a little extra motivation for myself to actually stick to a project and hold myself to account, after fearing myself to be slipping into being a bit rusty and outmoded in some of my dev work.

My goals at the moment are to:

  • Brush up on C# fundamentals and best practices, particularly when working with C# 10 & .NET 6.0.
  • Learn more about Blazor and make a few mini projects to get stuck into implementing various useful stuff.
  • Learn more about Docker and play around with containerisation.
  • Revisit 38Deputies.gg (a web app I developed to support the Guernsey 2020 election), write some posts about the architecture and features of the platform, and port it to some version of .NET/.NET Core, so that I can host it as a portfolio piece on a DigitalOcean droplet – currently, it’s not cost effective for me to keep it up while it’s stuck on Windows Server, and it’ll be a useful learning process to port it over.
  • Keep this blog up to date on all of the above and more, to keep motivating myself to learn and fully understand what I’m doing!

On top of the personal development side, I’ll be talking about things I already consider myself fairly well-versed in, or just things I find interesting. Hopefully by committing to all of the above, this post will stand the test of time amongst a good load of future blog posts, as proof that I have been true to my word!