patkua@work

The intersection of technology and leadership

Page 21 of 53

Poka-Yoke Your Developers

Poka-Yoke is a term coming from lean manufacturing and is about error-proofing, or preventing an error happening in the first place. A few weeks ago, we were working on fixing those flaky tests. A cause is the bad testing strategy, trying to set mocks on objects held by spring globaltons.

The previous developers had apparently known about this and went through diligently fixing each tests by ensuring that any test setting a mock was reset to its proper form. Unfortunately this knowledge wasn’t passed on to our team, and wasn’t obvious from the intent of the tests. More than that, it was easy to add new tests of this style and forget to reset the objects registered in spring.

Wanting to apply Poka-Yoka to our situation, we wrote something that would check the precondition that there were no side effects from other tests. Our implementation meant adding a @Before method to the base class these classes all shared that would cycle through all objects registered in spring, noting if any of their internal fields had been left mocked. If anything was detected, our assertion failed message contained a message that would explain why it was there and what to do about it. By adding this in, we actually found three existing tests that failed to clean up properly after themselves.

Poka-Yoka is fairly easy. When thinking about applying it, think about a solution that prevents the error happening more than once. Ideally this means that error is unavoidable (in our situation – this would mean changing the testing pattern across all tests), or at least by bare minimum, provide feedback when a person repeats the mistake.

Ruby Script to Capture HTTP traffic

I spent some time early this week trying to debug why some test was failing. It was using RESTeasy and I wanted to find out exactly what HTTP packets it was sending. If I was on a windows machine, I’d look towards Fiddler, but I couldn’t find anything easy on mac, so I wrote a little ruby script that I could point the client at and dump out all headers and request body.

It looks like this:

require 'webrick'
include WEBrick

class Simple < WEBrick::HTTPServlet::AbstractServlet
  def do_POST(request, response)
    puts "Body: " + request.body
    puts "Header: " + request.raw_header.to_s

    response.status = 200
  end
end

server = HTTPServer.new(:Port => 5899)
server.mount "/", Simple

['TERM', 'INT'].each do |signal|
trap(signal){ server.shutdown }
end

server.start

Update
Upon publishing this, I got a couple of alternatives worth looking into for next time.

  • Jim suggested using tcptrace; and
  • Gaz suggested using the following command sudo tcpdump -vvv -A -s 0 -i en1 tcp ‘port 5899’

This is just yet another example about why it’s important to have diversity when problem solving. I didn’t even know about these tools. Now I do and you do too.

Three Days of Clojure Joy

A couple of weeks ago, I sat in a training class run by Stuart Halloway, CEO of Relevance and who ThoughtWorks happened to bring across from the US to give us the low down of Clojure.

We covered a lot of material in the three days – covering pretty much every aspect of Clojure. We learnt all the basics such as core data types, namespaces, how to get documentation and examples, and many of the key functions that come as part of the core clojure libriares.

I appreciated the time spent discussing the background of why clojure exists, and the alternatives that are present in many other languages. It was great to have someone as knowledge as Stuart as we could ask as many questions as we like about things.

The training wasn’t just all talking and we did do a few exercises. I have to admit because I hadn’t touched any real functional programming since university, many of the exercises towards the end of the three days went over my head a little bit – a combination of needing to know some key clojure functions, and probably needing to re-adjust to a “functional way of thinking” which I think proves the biggest challenge.

In terms of tooling, we used the simple REPL and whatever environment we chose (the exercises don’t really need that much) although I learned about ParEdit that seemed to be quite useful.

I also learned about a new datastructure – Bit Partitioned Hash Trees (more here and here that I need to do some more reading around.

The course accelerated quickly, covering basic functional programming on the first day, the different ideas between state and time in clojure and then syntatic programming. Also we covered topics such as Interoperability with Java libraries and the problems that protocols address and defrecords (prefer defrecords over deftypes as the latter is now considered deprecated).

Just like any good training course, it left me realising how much more I need to learn and get my head around. This only really comes with more time and practice.

Beware the Spring Globaltons In Integration Tests

As I mentioned in a previous post, we’ve ben trying to fix a whole slew of tests that flicker, and we’ve spent some time fixing a number of integration tests that leave side effects.

This is the first time I’ve encountered the testing pattern (not recommended!) where SpringJUnit4ClassRunner loads up a spring context, a bean is pulled out, and then mocks used to stub out services. It’s a really evil pattern.

For one thing, mocks are about interaction based testing, not really about stubbing. I typically use them to drive out class-based roles. However, in this case, they were used to take out a portion of the application.

Using the above JUnit Runner means that there is one spring context per test run, effectively, a global pool of objects. Of course, when you pull one out and start modifying the objects, it means you have plenty of side effects across other tests. Unfortunately the don’t always manifest themselves in obvious manners.

Our solution to fix this was to use reflection to look at all beans in the spring context, and fail the test if it found any that had any mocked instances. I’d still recommend you avoid the testing pattern altogether, but if you are forced down this route, you now have a solution to detect side-effects across tests.

Book Review: “Get Better Faster: Ultimate Guide to Practicing Retrospectives”

I came across this free e-book, “Get Better Faster: Ultimate Guide to Practicing Retrospectives” via twitter (though I can’t seem to find the tweet that led me to it anymore) and thought I should write a review of it, like any other book. I’ll use a variation of the perfection game for this review.

Things I wouldn’t change
Firstly, it’s licensed under Creative Commons so anyone can use it. Yay. Then there’s the fact that it’s an eBook so it makes it quickly accessible to every. I guess it’s a bit more like giant slideware than a typical book, but that makes it easy to digest when reading on a computer – generally not to much text and nice little break out sections to help keep it varied.

The book references a number of interesting related practices such as After Action Review (AAR) developed by the US Army, though note they refer to it by the After Review Cycle that seems to be coined by a single consultancy. I will point out here that I disagree with the disadvantages of AAR that they list to compare them to retrospectives. If you read it, for instance, replace “ARC” with “Improvement” or “Learning” and it still reads the same. My point is that the inherent disadvantages are not an essential quality of the tool itself).

I like the 8 tips for better retrospectives that are widely useful for many teams and environments, mirroring many solutions to those I’ve suggested in my previous serious on, “When Retrospectives Go Wrong“. I like the way they highlight the importance of a facilitator role (which, in my experience, definitely helps make any important meeting more successful) and the provide some tips directly for the facilitator.

Finally they provide some good advice on making recommendations more tangible and, hopefully, more likely to be implemented.

Things that are different
Many people often ask the question why bother running retrospectives, and they have a page on metrics on retrospectives. I don’t necessarily agree with all of the metrics, it does answer some immediate questions managers and executives often have (unfortunately it is often the wrong question, in my experience, to answer).

Things that would make it perfect
It saddens me that, in the book, they make no reference, nor no acknowledge to other books on the topic. There is a small list of articles linked to some web resources, but I find it strange that they made no references to Kerth’s “Project Retrospectives” book, or Derby and Larsen’s “Agile Retrospectives” book. Nor to websites such as http://www.retrospectives.com or http://agileretrospectivewiki.org/

It also pains me the whole eBook is covered in excessive branding (twice on every page!) Fair enough they built it to market themselves, but really!

There is very good advice in the book, such as “set the meeting for one-to-three hours, with one hour often being ideal”. These novice rules/guidelines are important however I feel prevents people from achieving true mastery without explaining why these are the cases. In this example, the question best answered is, “Why is one hour ideal?” The lack of explanations is a problem as people who start to break these rules don’t know where to look next, or what to try, when the context isn’t appropriate.

Finding real object under spring proxies

As part of detecting side effects across tests, we needed to inspect some objects in tests autowired by spring. Unfortunately these objects were automatically proxied with the current configuration (and needed by some tests) so we needed some way of accessing the underlying proxied objects.

We didn’t really find many good examples of how to go about bypassing the spring proxies created through AOP. Doing a bit of debugging we found that spring used CGLib to generate its proxies and there is fortunately a method made accessible called getTargetSource() that gives you the underlying object. Here’s the helper class we wrote to extract it.

public class CglibHelper {
    private final Object proxied;

    public CglibHelper(Object proxied) {
        this.proxied = proxied;
    }

    public Object getTargetObject() {
        String name = proxied.getClass().getName();
        if (name.toLowerCase().contains("cglib")) {
            return extractTargetObject(proxied);
        }
        return proxied;
    }

    private Object extractTargetObject(Object proxied) {
        try {
            return findSpringTargetSource(proxied).getTarget();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    private TargetSource findSpringTargetSource(Object proxied) {
        Method[] methods = proxied.getClass().getDeclaredMethods();
        Method targetSourceMethod = findTargetSourceMethod(methods);
        targetSourceMethod.setAccessible(true);
        try {
            return (TargetSource)targetSourceMethod.invoke(proxied);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    private Method findTargetSourceMethod(Method[] methods) {
        for (Method method : methods) {
            if (method.getName().endsWith("getTargetSource")) {
                return method;
            }
        }
        throw new IllegalStateException(
                "Could not find target source method on proxied object [" 
                        + proxied.getClass() + "]");
    }
}

Clearing In-Memory Messages held in ActiveMQ

One of the major troubles with integration tests are the tradeoffs you get with speed and feedback. As systems get larger, the more integration tests you have, and the more chance you might preserve state. I’ve been fighting a build inherited by our team where intermittent integration tests prove troublesome. In the next series of posts, I’ll detail some of the issues we’ve found and some of the fixes.

Our current system uses JMS for messaging and one of the first stateful problems we found were messages being left over from previous test runs in our ActiveMQ instance. We had a few options for doing it out of memory including a purge script and a call to deleteAllMessages though the latter failed because we stored messages only in memory.

Instead we worked out how it stores messages, and wrote a class to go through and clean them all up.

public class ActiveMqHelper {

    public void removeAllMessages() {
        Map<String,BrokerService> brokers = BrokerRegistry
                .getInstance().getBrokers();
        try {
            for (BrokerService brokerService : brokers.values()) {
                Broker broker = brokerService.getBroker();
                new ActiveMQBrokerExtension(broker).clearAllMessages();
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    private class ActiveMQBrokerExtension {
        private final Broker broker;

        public ActiveMQBrokerExtension(Broker broker) {
            this.broker = broker;
        }

        public void clearAllMessages() throws Exception {
            Map<ActiveMQDestination, Destination> destinationMap 
                = broker.getDestinationMap();
            for (Destination destination : destinationMap.values()) {
                ActiveMQDestination activeMQDestination 
                        = destination.getActiveMQDestination();
                if (activeMQDestination.isTopic()) {
                    clearAllMessages((Topic)destination);
                } else if (activeMQDestination.isQueue()) {
                    clearAllMessages((Queue) destination);
                }
            }
        }

        private void clearAllMessages(Topic topic) throws IOException {
            List<Subscription> consumers = topic.getConsumers();
            for (Subscription consumer : consumers) {
                ConnectionContext consumerContext = consumer.getContext();
                MessageStore messageStore = topic.getMessageStore();
                messageStore.removeAllMessages(consumerContext);
            }
        }
        private void clearAllMessages(Queue queue) throws Exception {
            queue.purge();
        }
    }
}

Someone might one day benefit from this.

Drive by Dank Pink

I finally picked up a copy of Drive to read. The book looked ominously big, however I found out the two hundred (ish) pages had been printed on fairly thick paper and was pretty engaging to read overall. Pink focuses on a topic close to my heart, describing the way that people find engagement and what drives people. It’s quite relevant to the way that I like to work, and what the company I work for strives to achieve.

Pink covers a lot of interesting material, including many references and backed up by a lot of research. In it, he compares classic management techniques (financial incentives) and details research that describes why they fail to achieve what they need to do. He talks about interesting research that shows that a small financial incentive helps boost short term performance (in work that requires no thinking) at the cost of long term performance and detriment to creativity. For today’s information worker, and the chaordic environments we work in, this should really be raising alarm bells. To me, it’s akin to the systems thinkers that know that measuring and rewarding workers on the wrong incentives causes lots of problems. It also rings anecdotally to what I’ve seen some really talented people who work in these financial institutions driven into strange behaviours due to “bonus schemes” and loops.

Pink reiterates over points such as once we hit a certain wealth, happiness is no longer correlated with wealth. We seek, instead greater things. He gives many examples about how money isn’t a motivator for many people citing the achievements of Wikipedia, open source communities that build software such as Firefox and many more. He talks about once this is achieved, people are driven to solve problems, and be creative. He gives examples where people are driven to complete puzzles and games, not because they are paid to do it, because of some intrinsic drive.

Pink continually describes what gap there is between research and business and provides a way forward describing a number of elements necessary to satisfy this intrinsic drive. He talks about the need for autonomy, the idea of achieving mastery, and developing a true sense of purpose as well as providing a toolkit for people and managers to achieve this. I like to think that the agile values help businesses to focus on creating environments that help satisfy this new style of management. The idea of Software Craftsmanship emphasising lifelong learning and mastery, a common theme in agile teams to be autonomous in task completion and done right with lean thinking, building the right thing for a good purpose.

Above is a wonderful RSAnimate video helping to summarise the book.

« Older posts Newer posts »

© 2024 patkua@work

Theme by Anders NorenUp ↑