MySQL AES Encryption in Ruby

I was asked to find a way to encrypt strings in Ruby that would be compatible with MySQL's AES_ENCRYPT and AES_DECRYPT functions. I found several solutions online, but none of them worked the way I expected. After cobbling together several examples, I finally came up with a solution that proved to be compatible with the MySQL version. This was written for use with Ruby 2.3.5. This module can be used as follows:

Running a Tomcat-based Spring Boot application in Docker

In one of the projects I am working on, I was tasked from taking a Tomcat application that ran on an EC2 server, and get it running with the Spring Boot framework in a Docker container. The twist is that the application needed read-write access to the resources folder within the project. In order to do that, the application needed to be run as an "exploded" WAR file (It had been run that way on the old server as well).

I accomplished this with this Docker file:

The WAR file is compiled with the "spring-boot-maven-plugin", and includes all the dependency JARs, so the application can be run stand-alone.

That entry in the POM is:

And the "org.springframework.boot.loader.WarLauncher" Class is what Spring Boot uses to bootstrap the applicatiton on the embedded Tomcat.

OpenPojo Java 9 Compatibility Workaround

I am working on updating applications at work to Java 9. I have been using OpenPojo for years to test all my POJOs in one go. However, I found that the tests started throwing the exception:

java.lang.NoClassDefFoundError: Could not initialize class com.openpojo.reflection.java.packageloader.Package

I traced through the OpenPojo code and found that it was hard coded to read the Java class path from the old "sun.boot.class.path" system property. This property has been completely removed from Java 9, in favor of "java.class.path," which has been available since at least Java 7. (See https://docs.oracle.com/javase/8/docs/technotes/tools/windows/findingclasses.html)

I submitted my findings in an issue for the developers consideration: https://github.com/oshoukry/openpojo/issues/108

In the meantime, I developed the following workaround that can be inserted into a current POJO test class, and will allow the code to function the same way in Java 9.

Find All the Divisors for a Number

In some of my load testing, I want to run a certain number of transactions. My script takes parameters for a number of threads and the number of loops all the threads should go to. When I have a certain target number I want to get to, and an idea of what number of threads I want, I would like to figure out exactly how many loops I need to run for an exact number of threads.

For example, lets say I want to process 16,000,000 transactions with about 300 threads. I can run the following Java class, and figure out that if I use 320 threads, I need 50,000 loops.

Enter an integer: 16000000
Even divisors for 16000000
8000000 x 2 = 16000000
4000000 x 4 = 16000000
3200000 x 5 = 16000000
2000000 x 8 = 16000000
1600000 x 10 = 16000000
1000000 x 16 = 16000000
800000 x 20 = 16000000
640000 x 25 = 16000000
500000 x 32 = 16000000
400000 x 40 = 16000000
320000 x 50 = 16000000
250000 x 64 = 16000000
200000 x 80 = 16000000
160000 x 100 = 16000000
128000 x 125 = 16000000
125000 x 128 = 16000000
100000 x 160 = 16000000
80000 x 200 = 16000000
64000 x 250 = 16000000
62500 x 256 = 16000000
50000 x 320 = 16000000
40000 x 400 = 16000000
32000 x 500 = 16000000
31250 x 512 = 16000000
25600 x 625 = 16000000
25000 x 640 = 16000000
20000 x 800 = 16000000
16000 x 1000 = 16000000
15625 x 1024 = 16000000
12800 x 1250 = 16000000
12500 x 1280 = 16000000
10000 x 1600 = 16000000
8000 x 2000 = 16000000
6400 x 2500 = 16000000
6250 x 2560 = 16000000
5120 x 3125 = 16000000
5000 x 3200 = 16000000
4000 x 4000 = 16000000
3200 x 5000 = 16000000
3125 x 5120 = 16000000
2560 x 6250 = 16000000
2500 x 6400 = 16000000
2000 x 8000 = 16000000
1600 x 10000 = 16000000
1280 x 12500 = 16000000
1250 x 12800 = 16000000
1024 x 15625 = 16000000
1000 x 16000 = 16000000
800 x 20000 = 16000000
640 x 25000 = 16000000
625 x 25600 = 16000000
512 x 31250 = 16000000
500 x 32000 = 16000000
400 x 40000 = 16000000
320 x 50000 = 16000000
256 x 62500 = 16000000
250 x 64000 = 16000000
200 x 80000 = 16000000
160 x 100000 = 16000000
128 x 125000 = 16000000
125 x 128000 = 16000000
100 x 160000 = 16000000
80 x 200000 = 16000000
64 x 250000 = 16000000
50 x 320000 = 16000000
40 x 400000 = 16000000
32 x 500000 = 16000000
25 x 640000 = 16000000
20 x 800000 = 16000000
16 x 1000000 = 16000000
10 x 1600000 = 16000000
8 x 2000000 = 16000000
5 x 3200000 = 16000000
4 x 4000000 = 16000000
2 x 8000000 = 16000000
1 x 16000000 = 16000000
=== End ===

Essentially, this provides all the numbers that evenly divide the provided number. Or, in other words, find multipliers that will result in a desired product.

Cleaning Up the IntelliJ IDEA Clone Repository Dialogue Box

Over the course of the last couple of years, I have checked out dozens of projects with IntelliJ IDEA from various Git repositories. Many of these projects were one-time checkouts, and a slew of them are no longer valid because of domain name changes.

There is no internal mechanism to remove URLs from this dialogue box, and I had not found anything on the Internet on how to do this. I was certain that this information had to be in a configuration file somewhere.

I ran across this page: Directories used by the IDE, which headed me in the correct direction. I eventually found the file I was looking for (using Mac OS X):

~/Library/Preferences/<PRODUCT><VERSION>/options/vcs.xml

This file has a set of <UrlAndUserName> elements that can be individually deleted, as needed:


<application>
  <component name="GitRememberedInputs">
    <option name="visitedUrls">
      <list>
        …
        <UrlAndUserName>
          <option name="url" value="https://github.com/appium/sample-code.git" />
          <option name="userName" value="" />
        </UrlAndUserName>
        …
      </list>
    </option>
    <option name="cloneParentDir" value="$USER_HOME$/IdeaProjects" />
  </component>
</application>
 
Exit IntelliJ IDEA completely, and start it up again. The next time you use the Clone Repository dialogue box (e.g., using "Check out from Version Control" in the Welcome dialogue box), you will see the list reduced to whatever entries you left in the vcs.xml file.

This solution was tested in version 2017.1, and I confirmed the same file location for version 2016.1.

Kubernetes Readiness and Liveness with Apache Kafka REST Proxy

When setting up Readiness and Liveness checks in Kubernetes for Kafka connectors, the use of the httpGet described in my previous blog post (Kubernetes Readiness and Liveness with Spring Boot Actuator) is not an option because there is no endpoint to reference. These can be deployed with the Apache Kafka REST Proxy, which gets us on the right path, but doesn't quite work how we want in this respect.

The Kafka REST Proxy provides endpoints that allow one to get some basic status info about connectors. However, the standard Kubernetes httpGet calls use status code >= 200 and < 400 to determine the status, and since the Kafka REST status endpoint always provides a 200 status code, it is not possible to use this methodology to determine if a connector is down.

What we would like to do is check the content of the status call, and do a string comparison. For example, when the service is up, the status endpoint indicates that the state is "RUNNING":

# curl http://10.30.128.1:8083/connectors/mysql-kafka-connector/status

{"name":"mysql-kafka-connector","connector":{"state":"RUNNING","worker_id":"10.30.128.1:8083"},"tasks":[{"state":"RUNNING","id":0,"worker_id":"10.30.128.1:8083"}]}

We can pause the connector using this endpoint:

# curl -i -X PUT http://10.30.128.1:8083/connectors/mysql-kafka-connector/pause

HTTP/1.1 202 Accepted

And then the state is changed to PAUSED:

# curl http://10.30.128.1:8083/connectors/mysql-kafka-connector/status

{"name":"mysql-kafka-connector","connector":{"state":"PAUSED","worker_id":"10.30.128.1:8083"},"tasks":[{"state":"PAUSED","id":0,"worker_id":"10.30.128.1:8083"}]}


To accomplish this check, we can leverage the exec command probe:

readinessProbe: 

  exec: 

    command:

      - /bin/sh 

      - -c 

      - curl -s http://127.0.0.1:8083/connectors/mysql-kafka-connector/status | grep "RUNNING"

  initialDelaySeconds: 240

  periodSeconds: 5

  timeoutSeconds: 5 

  successThreshold: 1

  failureThreshold: 10 

livenessProbe:

  exec: 

    command: 

      - /bin/sh

      - -c 

      - curl -s http://127.0.0.1:8083/connectors/mysql-kafka-connector/status | grep "RUNNING"

  initialDelaySeconds: 300 

  periodSeconds: 60

  timeoutSeconds: 10 

  successThreshold: 1

  failureThreshold: 3

The the exec command allows us to execute a shell command. In this case:

• running the shell (/bin/sh)
• telling it to run a single command (-c)
• with the command being a cURL call to the specific connector status endpoint, and grepping for the string "RUNNING"

When the grep is successful, Kubernetes interprets this as a success. If the grep comes back empty (i.e., "RUNNING" is not found), then it gets viewed as a failure. You can then test this on the server by pausing the service in question as described above.

To get the service running again and start passing readiness and liveness again, then you will want to use the RESUME endpoint.

# curl -i -X PUT http://10.30.128.1:8083/connectors/mysql-kafka-connector/pause

HTTP/1.1 202 Accepted

Kubernetes Readiness and Liveness with Spring Boot Actuator

In Kubernetes, "readiness" is the indicator that the service is ready to accept traffic, and is only performed at the beginning of a pod's life cycle. "Liveness" is a periodic health check that should indicate that the service is still functional within the pod.
More details can be found in the documentation at
https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/

Spring Boot is a stand-alone Spring environment well suited for micro services.
https://projects.spring.io/spring-boot/

Spring Boot has a bundle available called "Actuator" that exposes several helpful endpoints, one of which is a "health" endpoint. The simple version of this endpoint returns a simple JSON with a status UP or DOWN.
https://github.com/spring-projects/spring-boot/tree/master/spring-boot-actuator

One particular feature of the health endpoint that is useful is that besides the text indicator in the JSON response, it also signals up and down through the status code. UP = 200, and DOWN = 503 (Service Unavailable).

Putting this all together, the readiness and liveness configuration in the Kubernetes deployment YAML can look something like this:

readinessProbe:
httpGet:
  scheme: HTTP
  path: /health
  port: 8080
initialDelaySeconds: 240
periodSeconds: 5
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 10
livenessProbe:
httpGet:
  scheme: HTTP
  path: /health
  port: 8080
initialDelaySeconds: 300
periodSeconds: 60
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 3

In this example, we instruct Kubernetes to wait 240 seconds to allow the application to start before performing the check. It will retry up to 10 times with a five second pause between each try. It will wait for a maximum of 5 seconds for a result to be returned. After 10 failures, the pod will be restarted.

For the liveness check, we instruct Kubernetes to wait 300 seconds, and to check every 60 seconds. If three failures occur in a row, the pod will be restarted.

A toString Comparator for arbitrary objects

I recently ran into a problem where I wanted to add objects attained by reflection to a TreeSet. In this specific case, I had a problem when a Locale object was retrieved. It could not be added to the TreeSet because it is not "Comparable." It did, however, have toString values that could be sorted intelligibly.

I decided to create a Comparator that would handle arbitrary objects, and allow them to be added to a TreeSet of type Object:

The caveat is that this works best when the object in question overrides the default toString method in a meaningful way. If the base Object.toString() method is inherited by the object, then the sorting will most likely appear random, since the method looks like this:

    public String toString() {
        return getClass().getName() + "@" + Integer.toHexString(hashCode());
    }

Fix for "HibernateJpaDialect - JDBC Connection to reset not identical to originally prepared Connection" warning

After updating an application to Hibernate 5.1.1 from 5.1.0, I started seeing the following warning:

HibernateJpaDialect - JDBC Connection to reset not identical to originally prepared Connection

I did not find an "explicit" solution on the Internet, but looking through some of the code fixes in the ngrinder project, I figured out that I could fix my problem the same way by adding <prop key="hibernate.connection.release_mode">on_close</prop> to applicationContext.xml:

Thanks for this solution, JunHo!

JMeter in IntelliJ IDEA on Mac OS X

Quick Answer

Manually add the JMeter path at

Preferences » Other Settings » JMeter » JMeter home directory

Long Answer

I wanted to be able to launch JMeter scripts from within IntelliJ IDEA. This is the process I went through to make this happen:

Install JMeter

Having already installed Homebrew, JMeter can be easily installed with the command

brew install jmeter

Install JMeter Plugin in IntelliJ IDEA

Preferences » PlugIns » Search for JMeter » Select JMeter plugin » install » restart IntelliJ

Configure JMETER_HOME

This is where the troubleshooting needed to start. For the quick solution, see the first section. Otherwise, this is the process I went through:

Ideally, one would want things to "just work" after installing JMeter and the plugin. Unfortunately, when I created my first JMeter Run Configuration in IntelliJ, I got the message in the dialog box:

Run Configuration Error: JMeter not found

I eventually found a dialogue box under

Preferences » Other Settings » JMeter

with a helpful indicator that

JMETER_HOME is used by default

Sure enough, when I ran the env command in a Terminal, JMETER_HOME was not there.

To find out what my JMeter home directory was, I did the following in the Terminal

$ which jmeter
/usr/local/bin/jmeter
$ cat /usr/local/bin/jmeter
#!/bin/bash
exec "/usr/local/Cellar/jmeter/3.0/libexec/bin/jmeter" "$@"

So, based on some other information I gathered, the home directory was just above the "bin" directory.

My first attempt was to add the following line to my .bash_profile:

export JMETER_HOME=/usr/local/Cellar/jmeter/3.0/libexec

Sadly, I found this didn't work after rebooting and confirming that it showed up when the env command was executed in the Terminal.

I eventually ran across Setting global environment variables in IntelliJ IDEA and other test config goodies, from which I learned that environment variables are not automatically passed to GUI applications in Mac OS X. Nonetheless, Update 2 in the article looked promising.

I then added this line to .bash_profile:

launchctl setenv JMETER_HOME /usr/local/Cellar/jmeter/3.0/libexec

Again, to my frustration, I found that after rebooting and confirming that launchctl getenv JMETER_HOME displayed the correct value in the terminal, that this only worked in IntelliJ when I started IntelliJ, quit completely, and then started it again. I have no idea why the environment variable only seems to get read on the second startup.

So, for my final answer, I just had to go with the manual solution of adding the JMeter home path as an Override in the dialogue box found at:

Preferences » Other Settings » JMeter » JMeter home directory

Extract the icon from an APK file

Using the apk-parser library (https://github.com/caoqianli/apk-parser), I developed the following code that will extract the icon from an APK file. Note that there is no guarantee about what size the icon will be. Enjoy!

Renaming a dependency JAR before buiding a WAR in Maven

I was recently tasked with updating New Relic on our servers. When this was initially set up, the server was set up with a JVM command line option:

-javaagent:/usr/share/tomcat7/webapps/ROOT/WEB-INF/lib/newrelic-3.9.0.jar

When working on updating the version, I wanted to change this so that

1. We could update the New Relic Agent whenever a new update is available
2. We would not need to change the JVM command line each time the Agent was updated

To accomplish this, I wanted the New Relic Agent JAR name to always be the same, regardless of the version. I found maven-dependency-plugin through a web search, but ran across a problem where my WAR file was being created before the JAR was downloaded and renamed (using maven-war-plugin). All the examples I ran across used <phase>package</phase>, and while that looked right, I figured that this was still happening in the wrong life cycle phase for what I wanted to do. I resorted to reading the documentation, and with some experimenting found that <phase>prepare-package</phase> did this at the right part of the life cycle.
Here is the plugins section of the working pom.xml:
You do not need to include this dependency in your dependencies section, unless you are actually using it in your code.
We changed the server's JVM command line option to:

-javaagent:/usr/share/tomcat7/webapps/ROOT/WEB-INF/lib/newrelic.jar

And everything worked just great ... once I got the YAML file figured out 😉.

---

Bonus Notes

• If you have not worked with YAML before, you will find out quickly that indentation is important to keep the hierarchy of properties right
• The newrelic.yml file included as an example in the newrelic-java.zip file appears to have an error in it. Specifically, the classloader_excludes property values need to be a commented list on the same line. I got parse errors using the example as-is (i.e., the list is indented with each item on a separate line, and has an extra comma at the end).
• If you are reading this and setting up a new configuration based on this article, you will also need the newrelic.yml file to end up in the same folder. To that end, place the file in /src/main/webapp/WEB-INF/lib in your Maven-based folder structure.

IntelliJ Idea: Can't find gems in Cucumber configurations when using RVM Ruby

I had updated Ruby on my Mac OS X laptop using RVM:

curl -sSL https://get.rvm.io | bash -s stable --ruby
rvm use 2.3 --default

In IntelliJ IDEA Ultimate 2016, I changed my Cucumber configuration to use the "RVM: ruby-2.3.0" SDK.

I then got any number of errors regarding not having gems installed like cucumber, any of the required gems, and then finally the debug gems (ruby-debug-ide and debase).

Run Configuration Error: Cucumber Gem isn't installed for RVM

Initially, I had some success getting rid of errors one by one by manually running "gem install" on the command line for every gem that was missing. However, in the end, I still had the problems with the debug gems not being installed, and getting errors when attempting to have IntelliJ install the debug gems itself.

I finally figured out that the Gems bin directory was incorrect. When I when to

File ➜ ProjectStructure ➜ SDKs ➜ RVM: ruby-2.3.0

and changed Gems bin directory to

/Users/[username]/.rvm/gems/ruby-2.3.0

Then things started working just fine.

Finagle Filter path with "andThen"

In way of passing on what I am continuing to learn about chained Finagle Filters in Scala:

Filters can be chained together with the “andThen” function. This is essentially an indicator of which direction the Request (input) is handed off to the next filter. I believe that when we normally think about filters, we expect the filter to act on the Request (like a sieve, for example), and indeed it can. However, once the Request gets to the end of the Filter chain, it gets turned into a Response (output), which also, in turn, can be filtered as it is passed back back to the beginning of the Filter chain.

Here is a ScalaTest that shows how both the inward and outward paths can be used to modify the request and the response, as well as a short-circuit in Filter3 that prevents Filter4 from being run (you can change the condition to true to see the path through all four filters). The example Finagle Service here simply takes an initial value and concatenates the request to make a Response:

import com.twitter.finagle.{Service, SimpleFilter}
import com.twitter.util.Future
import org.scalatest._

class StringService(response: String) extends Service[String, String] {
  override def apply(request: String): Future[String] = Future.value(response + ":" + request)
}

object StringFilter1 extends SimpleFilter[String, String] {
  override def apply(request: String, service: Service[String, String]): Future[String] = {
    val requestUpdate = request.concat(" » enter-1")
    service(requestUpdate).map(futureString => futureString.concat(" » exit-1"))
  }
}

object StringFilter2 extends SimpleFilter[String, String] {
  override def apply(request: String, service: Service[String, String]): Future[String] = {
    val requestUpdate = request.concat(" » enter-2")
    service(requestUpdate).map(futureString => futureString.concat(" » exit-2"))
  }
}

object StringFilter3 extends SimpleFilter[String, String] {
  override def apply(request: String, service: Service[String, String]): Future[String] = {
    val requestUpdate = request.concat(" » enter-3")
    val myCondition = false
    if(myCondition){
      service(requestUpdate).map(futureString => futureString.concat(" » exit-3"))
    } else {
      Future(requestUpdate.concat(" » short-circuit-3"))
    }
  }
}

object StringFilter4 extends SimpleFilter[String, String] {
  override def apply(request: String, service: Service[String, String]): Future[String] = {
    val requestUpdate = request.concat(" » enter-4")
    service(requestUpdate).map(futureString => futureString.concat(" » exit-4"))
  }
}

class FilterStackTest  extends FlatSpec with Matchers {
  "A Filter" should "Operate like a Stack" in {
    var testService =  new  StringService("Service A")
    var testFilter = StringFilter1 andThen StringFilter2 andThen StringFilter3 andThen StringFilter4

    System.out.println(testFilter("start",testService))
  }
}

The console output is:
Promise@71098046(state=Done(Return(start » enter-1 » enter-2 » enter-3 » short-circuit-3 » exit-2 » exit-1)))

And when val myCondition = true :

Promise@380962452(state=Done(Return(Service A:start » enter-1 » enter-2 » enter-3 » enter-4 » exit-4 » exit-3 » exit-2 » exit-1)))

Google API Fusion Table permissioning

I recently worked on a project where I needed to update an application that leveraged Google Fusion Tables. The Google API changed significantly, and the application did not work anymore. While I found a good Fusion Table coding example of how to get the Java code changed properly, I had a lot of difficulty getting the permissioning set up.

Here is a brief summary of how I got it to work, in the hopes that it might help others who are having similar problems:

Connect the table to the Fusion Table application

• If the table you are interested in is not already connected to Fusion Tables, click it in your Google Drive, and then click the Connect button.

Turn on the Fusion Table API Service

• Open the Google API Console
• Create a new project if you need to
• In Services, turn on Fusion Tables API

Set up a Service Account

• In the Google API Console, open API Access and click the Create an OAuth 2.0 client ID button
• Enter a Product Name and click Next
• Click the Service Account radio button, and then Create Client ID
• Download the key file into your project, and rename it to whatever is appropriate for you to use in your application
• You will also need the "Email Address", which is referred to as the "Application ID" within the API

Set permissions on the table file

This one was really difficult to figure out. If you need to do INSERTs or DELETEs into the Fusion Table, then you will need to set "writer" permissions for the Service Account. If you only need to SELECT from your application, then you can skip this step, of course.

• Open the Google Drive SDK Permissions Page
• Turn on the Authorize requests using OAuth 2.0 toggle (you should be prompted to authorize)
• Enter the following information:
  • Field: [the fusion table ID]
  • role: writer
  • type: user
  • value: ["Email Address" from the Console]

Java code

That should be it. Following the example code, you will set up a credential:

credential = new GoogleCredential.Builder()
    .setTransport(HTTP_TRANSPORT)
    .setJsonFactory(JSON_FACTORY)
    .setServiceAccountId(config.getAccountId())
    .setServiceAccountScopes(Collections.singleton(FusiontablesScopes.FUSIONTABLES))
    .setServiceAccountPrivateKeyFromP12File(keyFile)
    .build();

Make your Fusion Table object:

fusiontables = new Fusiontables.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential).build();

Run your various SQL statements:

response = fusiontables.query().sql(insertSql).execute();

Hope that helps someone!

BigDecimal.equals() vs. compareTo() Performance Test

Problem

There is much debate online about the merits of using the BigDecimal.equals(Object) method. In the equals method, scale is considered in such a way that 1.0 != 1.00. The compareTo method, however, ignores scale, meaning that 1.0 == 1.00. In this specific case, however, scale is known, so either method will work. Out of curiosity, I wondered if there was a performance hit in either method.

Approach

import java.math.BigDecimal;
import java.math.MathContext;
import java.math.RoundingMode;
import java.security.SecureRandom;

import org.joda.time.DateTime;
import org.junit.Test;

/**
 * <p>
 * Class to test the performance of {@link BigDecimal#equals(Object)} vs.
 * {@link BigDecimal#compareTo(BigDecimal)}. The question at hand is which
 * method to use when the precision and scale of the BigDecimals being compared
 * is know to be the same.
 * </p>
 * <p>
 * In the equals method, scale is considered in such a way that 1.0 != 1.00. The
 * compareTo method, however, ignores scale, meaning that 1.0 == 1.00. In this
 * specific case, however, scale is known, so either method will work. Out of
 * curiosity, I wondered if there was a performance hit in either method.
 * </p>
 */
public class BigDecimalPerformanceTest {

    /** The {@link SecureRandom} object. */
    public static final SecureRandom RANDOM = new SecureRandom();

    /** The number of times to loop over each comparison. */
    public static final int ITERATIONS = 100000000;

    /** The number of times to loop over all comparisons. */
    public static final int LOOPS = 3;

    /** BigDecimal precision. */
    private static final int PRECISION = 15;

    /** BigDecimal scale. */
    private static final int SCALE = 2;

    /** Equals vs. compareTo. */
    @SuppressWarnings("unused")
    @Test
    public void equalsVsCompareTo() {
        for (int j = 0; j < LOOPS; j++) {
            /* Reference object to do all the comparisons against. */
            BigDecimal testBigDecimal = makeBigDecimal();

            /*
             * Classic equals method. Assigning results to a variable for
             * consistency.
             */
            DateTime startTime = new DateTime();
            for (int i = 0; i < ITERATIONS; i++) {
                boolean test = testBigDecimal.equals(makeBigDecimal());
            }
            DateTime endTime = new DateTime();
            System.out.println("BigDecimal.equals()    : "
                               + endTime.minus(startTime.getMillis()).toString("m:s.SSS"));

            /* Plain compareTo method. */
            startTime = new DateTime();
            for (int i = 0; i < ITERATIONS; i++) {
                int test = testBigDecimal.compareTo(makeBigDecimal());
            }
            endTime = new DateTime();
            System.out.println("BigDecimal.compareTo()1: "
                               + endTime.minus(startTime.getMillis()).toString("m:s.SSS"));

            /*
             * Convert compareTo to true/false as a closer comparison with
             * equals as far as actual usage would be concerned.
             */
            startTime = new DateTime();
            for (int i = 0; i < ITERATIONS; i++) {
                boolean test = testBigDecimal.compareTo(makeBigDecimal()) == 0;
            }
            endTime = new DateTime();
            System.out.println("BigDecimal.compareTo()2: "
                               + endTime.minus(startTime.getMillis()).toString("m:s.SSS"));
        }
    }

    /**
     * Make big decimal.
     * 
     * @return the big decimal
     */
    private BigDecimal makeBigDecimal() {
        return new BigDecimal(RANDOM.nextDouble(), new MathContext(PRECISION, RoundingMode.HALF_UP))
                .setScale(SCALE, RoundingMode.HALF_UP);
    }
}

Output

BigDecimal.equals()    : 3:42.909
BigDecimal.compareTo()1: 3:44.995
BigDecimal.compareTo()2: 3:49.476
BigDecimal.equals()    : 3:48.313
BigDecimal.compareTo()1: 3:42.152
BigDecimal.compareTo()2: 3:43.889
BigDecimal.equals()    : 3:42.079
BigDecimal.compareTo()1: 3:44.639
BigDecimal.compareTo()2: 3:42.564

Conclusion

When you know the scale of a BigDecimal will be consistent, equals() and compareTo() perform equally as well. In my case, I chose to stick with using equals(), as it is immediately obvious in the code what I am trying to accomplish.

Bin2Txt - Remove Non-printable Characters from a Text File with Java

I recently was dealing with some DB2 "unload" (e.g., export) files that I wanted to parse and then load into Oracle. I found that the unload files use a lot of binary characters, which makes it very difficult to parse. I wrote the following Java class to convert the unprintable characters into a tilde (which is a character that does not occur in the data). This resulted in DB2 unload files that were parsable as fixed-width data files.
The main problem this approach does not attempt to solve is that the DB2 unload files save numeric fields as the actual value, not the digit equivalent (i.e., the number 84 is unloaded as the ASCII-equivalent "T", not "84"). This code obviously does not reference the DB2 "punch" (e.g., parse instruction) files, so it makes no attempt to parse the files into fields itself - that is a separate exercise in my case. BTW, if there is a good way to import these files into Oracle automatically, please let me know, as I have not been able to find a better solution.
This code is fairly generic, and can be used for other purposes beyond converting DB2 unload files, so if you have a need to replace non-printable characters in text files, you can start with this code base.

---

package com.threeleaf.bin2txt;

import java.io.File;

import java.io.FileInputStream;

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.InputStream;

import java.io.OutputStream;

/**

 * Purpose is to read a file and replace non-printable characters with a given character.

 * Specifically, I want to use this to make DB2 unload files parsable with other applications so

 * that the data can be imported into Oracle.

 *

 * @author John A. Marsh

 * @since 2011-10-27

 */

public final class Bin2Txt {

    /**

     * Run this class from the command line with:

     * java Bin2Txt <pathAndFilename>.

     *

     * @param args

     *        the filename to convert

     * @throws IOException

     *         Signals that an I/O exception (e.g., file not found) has occurred.

     */

    public static void main (final String[] args) throws IOException {

        final byte ASCII_SPACE = 32;

        final byte ASCII_CR = 13;

        final byte ASCII_LF = 10;

        final byte ASCII_TILDE = 126;

        try {

            final File file = new File(args[0]);

            final InputStream inputStream = new FileInputStream(file);

            final long fileLength = file.length();

            /*

             * Array needs to be created with an int type, so need to check to ensure that file is

             * not larger than Integer.MAX_VALUE.

             */

            if (fileLength > Integer.MAX_VALUE) {

                throw new IOException("File is too big");

            }

            /* Create the byte array to hold the data */

            final byte[] bytes = new byte[(int) fileLength];

            /* Read in the bytes */

            int offset = 0;

            int numRead = 0;

            while (offset < bytes.length && (numRead = inputStream.read(bytes, offset, bytes.length - offset)) >= 0) {

                offset += numRead;

            }

            /* Ensure all the bytes have been read in */

            if (offset < bytes.length) {

                throw new IOException("Could not completely read file " + file.getName());

            }

            inputStream.close();

            for (int i = 0; i < bytes.length; i++) {

                if (bytes[i] == ASCII_CR && bytes[i + 1] == ASCII_LF) {

                    /*

                     * Preserve line breaks (carriage return + line feed) by skipping over them.

                     * Note that I don't check for end of file here because I already know my

                     * particular files will never end with a CRLF.

                     */

                    i = i + 2;

                }

                if (bytes[i] < ASCII_SPACE || bytes[i] > ASCII_TILDE) {

                    /* Replace all non-printable characters. */

                    bytes[i] = ASCII_TILDE;

                }

            }

            /* Output file name will be the same as the input, with ".out.txt" added to the end. */

            final OutputStream outputStream = new FileOutputStream(args[0] + ".out.txt");

            outputStream.write(bytes);

            outputStream.close();

        } catch (final ArrayIndexOutOfBoundsException e) {

            /*

             * If no file was passed on the command line, this exception is generated. A message

             * indicating how to the class should be called is displayed.

             */

            System.out.println("Usage: java Bin2Txt filename\n");

        }

    }

}

---

Here is a batch file that will convert all the files in a given directory:

---

:: Directory where Bin2Txt.class is located ::

cd C:\projects\workspace\bin2txt\bin\

:: Put in directory where unload files are ::

for %%f in ("C:\projects\Database\Unloads\*.txt") do call java com.threeleaf.bin2txt.Bin2Txt %%f

---

Solved: Firefox Tab Icons Are Missing

I have been frustrated over the last few weeks because I could not figure out why the icons suddenly stopped showing up in Firefox. I have scoured the web and tried several suggestions, but none of them worked for me. I finally got clued in that it might be one of my extensions, and sure enough, going to Help » Restart With Add-ons Disabled caused the icons to reappear. After a little trial and error I finally found that it was the Favicon Picker 2. I wanted the functionality, so I tried Favicon Picker 3, but it caused the same problem. I finally found that Bookmark Favicon Changer (https://addons.mozilla.org/en-US/firefox/addon/bookmark-favicon-changer/) allowed me to customize my bookmark icons without causing the tabs to go blank. So, if you find that the favicons in your tabs are absent, give it a try. As of this writing, I am using Firefox 7 and Bookmark Favicon Changer 1.54.