The intersection of technology and leadership

Category: Development (Page 15 of 18)

Appreciating language features

Developing systems takes a very different outlook than it does from developing libraries, and again very different from designing languages. Even though on projects when we have specific coding standards, there’s always often a benefit to supporting much more. I appreciate language designers needing to think at a much broader scale about the realm of possibilities than I normally need to for systems I develop.

My example of this appreciation is when I had to debug some java code via a remote terminal on a box where the unwanted behaviour emerged. I was actually thankful for being able to do an import java.util.* rather than having to specify every single class that I wanted to use.

Generating a single fat jar artifact from maven

When using the jar-with-dependencies descriptorRef for the maven-assembly-plugin, it creates two files by default, the normal default jar with any library dependencies excluded, and a second jar with all the libraries included appended with “jar-with-dependencies”. Since I find maven help guides unintuitive, it took a while before we found the appendAssemblyId option to turn it off. Here’s the snippet of the pom.xml to create just a single far jar.

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <configuration>
    <appendAssemblyId>false</appendAssemblyId>
    <descriptorRefs>
      <descriptorRef>jar-with-dependencies</descriptorRef>
    </descriptorRefs>
  </configuration>
  <executions>
    <execution>
      <id>make-assembly</id>
      <phase>package</phase>
      <goals>
        <goal>assembly</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Maven-assembly-plugin ignoring manifestEntries?

We’re using this Maven plugin to generate a fat jar for a utility, effectively including all library dependencies un-jarred and re-jarred into a single distribution. The first part was easy, hooking the assembly goal of the maven-assembly-plugin onto the package goal in the maven build lifecycle. Our pom.xml had this entry in it

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <configuration>
    <descriptorRefs>
      <descriptorRef>jar-with-dependencies</descriptorRef>
    </descriptorRefs>
    <archive>
      <manifest>
        <mainClass>com.thekua.maven.ExampleProgram</mainClass>
      </manifest>
    </archive>
  </configuration>
  <executions>
    <execution>
      <phase>package</phase>
      <goals><goal>assembly</goal></goals>
    </execution>
  </executions>
</plugin>

We tried adding in our own entry into the manifest, the CruisePipelineLabel, with a value that should be set by Cruise. We added the new section so our pom.xml now looked like this:

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <configuration>
    <descriptorRefs>
      <descriptorRef>jar-with-dependencies</descriptorRef>
    </descriptorRefs>
    <archive>
      <manifest><mainClass>com.thekua.maven.ExampleProgram</mainClass></manifest>
        <manifestEntries>
          <CruisePipelineLabel>
            ${env.CRUISE_PIPELINE_LABEL}
          </CruisePipelineLabel>
        </manifestEntries>
    </archive>
  </configuration>
  <executions>
    <execution>
      <phase>package</phase>
      <goals><goal>assembly</goal></goals>
    </execution>
  </executions>
</plugin>

After running the target and inspecting the manifest.mf, I couldn’t see the additional property set. I did some searching, found a bug apparently fixed in the 2.2-beta-2 version. After some debugging, I found out that the plugin apparently does not include these additional entries if the value is not set. I tested this out by changing the line to:

<manifestEntries>
  <CruisePipelineLabel>aTestValue</CruisePipelineLabel>
</manifestEntries>

So the answer to whether or not maven-assembly-plugin ignores an element in the manifestEntries is to ensure the value is set before testing it. It looks like a null value is interpreted as “don’t include”.

Automated story-based acceptance tests lead to unmaintainable systems

Projects where the team directly translates story-level acceptance criteria into new automated test cases set themselves up for a maintenance nightmare. It seems like an old approach (I’m thinking WinRunner-like record-play back scripts), although at least the teams probably feel the pain faster. Unfortunately not many teams seem to know what to do. It sounds exactly like the scenarios that my colleagues, Phillip and Sarah are experiencing or experienced recently.

Diagnosing this style of testing is easy. If I see the story number or reference in the title of the test class or test case name, chances are, your team experiences automated story-based acceptance tests.

Unfortunately the downfall to this approach has more to do with the nature of stories than it does with the nature of acceptance tests (something I’ll get to later). As I like to say, stories represent the system in a certain snapshot of time. The same thing that lets us deliver incremental value in small chunks just doesn’t scale if you don’t consolidate the new behaviour of the system, with its already existing behaviour. For developers, the best analogy is like having two test classes for a particular class, one that reflected the behaviours and responsibilities of the system at the start, and one that represents the new behaviours and responsibilities of the system right now. You wouldn’t do this at a class level, so why should you do it at the system level?

Avoid temporal coupling in the design of your tests. The same poor design principle of relating chunks of code together simply because someone asked for them at the same time, also apply to how you manage your test cases. In terms of automated story-based acceptance tests, avoid spreading similar tests around the system just because they were needed at different times.

What is a better way? A suggested antidote…

On my current project, I have been lucky enough to apply these concepts early to our acceptance test suites. Our standard is to group tests, not based on time, but on similar sets of functionality. When picking up new stories, we see if any existing tests need to change, before adding new ones. The groupings in our system are based on the system level features, allowing us to reflect the current state of the system as succinctly as possible.

VDK all the way

After lots of research, I’m in favour of the Virtual Disk Driver program called VDK available from http://chitchat.at.infoseek.co.jp/vmware/vdk.html and distributed under the GPL licence. Here’s some of the features that attracted me to it:

  • VDK doesn’t require running an MSI for installation – In keeping with the principles of starting a new project (checkout and go), I can simply include this file as part of a code repostiory, and know that I’m not forcing every other developer yet another manual step. Although you do need to install VDK, it’s a command line registration that is as simple as: vdk.exe install.
  • VDK is fully executable via the command line – This allows me to wrap this in the language of my choice to help automate environment setup as needed for a particular test to run.
  • VDK supports VMDK (VMWare Disk) images – Using other tools to generate the files that I want, I can easily use VDK to mount them to a particular drive letter.
  • VDK supports mounting with different options – Including read-only, read-write, and write-block mode.
  • VDK has good documentation – I found it really easy to understand what commands to execute to install, remove, mount, unmount, all with different options because both the command line (vdk.exe help) and the readme.txt had plenty of information and examples. It also helps that it follows conventions with other command line programs (following the DOS conventions of parameters with slashes)
  • VDK is realiable – I did plenty of different tests mounting and unmounting and it just keeps going (though your Windows Explorer may need a refresh (F5) to keep up).

Using a VMWare Disk Image created by QEMU as discussed previously, I can now create a new virtual disk mounted in windows simply by using the following commands:

vdk install
vdk open 0 floppy_disk.vmdk /P:0 /RW /L:Y
format /fs:fat y:

See the image below:

VDK Example

RamDisk utilities for Windows

If you can’t tell from the latest series of posts, I’ve been working very closely with the file system, especially exceptional circumstances when certain conditions go wrong. I’ve found it really interesting to see how to automate some of the test scenarios since I’ve never done anything like this before. One of the things I’ve been looking at closely are RamDisk utilities for windows, especially for things I can manipulate programmatically.

Most of the utilities on the net seem really robust, with the most favoured being the freeware RamDisk that comes from MyDigitalLife. There are also plenty of other commercial alternatives. The freeware RamDisk has a simple user interface that makes it really easy to manually mount a drive though their instructions for command line execution must less user-friendly, with few examples available, and what help document there is very unintuitive. Even obvious attempts with class command line options (slashes, dashes, double dashes, colon-separate name-value pairs) failed and I ran out of patience.

Unfortunately most other RamDisk tools for windows are all GUI-based and, in general, cost some money to use.

Running tests on a specific OS under JUnit4

Our current development team is split, some working on windows machines, others on macs, with continuous integration tests running against both windows and linux environments. We’ve had a need to run operating specific JUnit4 tests and I got a little tired putting guard clauses into different @Before, @After and @Test methods to prevent a particular block of code running.

Using the new org.junit.runner.Runner annotation, and SystemUtils from commons-lang, here is the result:

package com.thekua.java.junit4.examples;

import org.apache.commons.lang.SystemUtils;
import org.junit.internal.runners.InitializationError;
import org.junit.internal.runners.JUnit4ClassRunner;
import org.junit.runner.notification.RunNotifier;

public class RunOnlyOnWindows extends JUnit4ClassRunner {

    public RunOnlyOnWindows(Class klass) throws InitializationError {
        super(klass);
    }

    @Override
    public void run(RunNotifier notifier) {
        if (SystemUtils.IS_OS_WINDOWS) {
            super.run(notifier);            
        }
    }
}

Our test classes, then look like this:

...
@RunWith(value=RunOnlyOnWindows.class)
public class WindowsOnlySpecificFooBarUnitTest {
    ...
}

Of course, you can use any other particular variations like SystemUtils.IS_OS_UNIX, or SystemUtils.IS_OS_MAC but we haven’t needed to yet.

Of course, this is easily turned into some sort of conditional runner abstract class but at least you get the basic idea.

java.io.File setReadonly and canWrite broken on Windows

File.setReadonly(true) doesn’t actually work on windows (at least on the JDK that we’re working on). According to the bug report filed here, it is just setting the DOS flag that prevents it from being deleted just not being written to.

Meanwhile, assuming you have mounted a read only disk partition in windows to, say X:, new File(“X:/”).canWrite() returns true when it really should be returning false. I’m still trying to find the bug reported to sun for this, or will later update this entry to include it. It seems to persist when running against both jdk1.5.0_15 and jdk1.6.0_07.

VMWare Disk Minimum File Size

The current versions of both VMWare Fusion and VMWare Player limit the minimum capacity of virtual disks (*.vmdk) to 100MB (or 0.1GB). Even their command line utility for windows (vmware-vdiskmanager.exe) describes this limit despite the file format allowing other file sizes to be specified (see screenshot below)

VMWare File Minimum Size

If you want to create a smaller disk, then I suggest downloading the QEMU program that allows you to create VMWare compatible disk images. When you install QEMU (at least for windows), you should find a qemu-img.exe file in their folder that gives you the ability to create your disk image. Here’s the command we used to create a 1MB VMware compatible image.

qemu-img create -f vmdk smallfile.vmdk 1M

Executing native processes in Java on Windows

This is probably something that old timers in Java will probably know, so I’m posting this more for my reference. Trying to execute:

Runtime.getRuntime().exec("dir");

results in the following stack trace.

java.io.IOException: CreateProcess: dir error=2
  at java.lang.ProcessImpl.create(Native Method)
  at java.lang.ProcessImpl.(ProcessImpl.java:81)
  at java.lang.ProcessImpl.start(ProcessImpl.java:30)
  at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)
  at java.lang.Runtime.exec(Runtime.java:591)
  at java.lang.Runtime.exec(Runtime.java:429)
  at java.lang.Runtime.exec(Runtime.java:326)

After doing some reading

error=2

apparently means file not found. The fix for something like this is to first pass everything to the windows command line shell (cmd.exe on windows xp). This seems to do the job better:

Runtime.getRuntime().exec("cmd /c dir");

The slash-C means “Carries out the command specified by string and then terminates”.

« Older posts Newer posts »

© 2024 patkua@work

Theme by Anders NorenUp ↑