Cleaning Up the IntelliJ IDEA Clone Repository Dialogue Box

Over the course of the last couple of years, I have checked out dozens of projects with IntelliJ IDEA from various Git repositories. Many of these projects were one-time checkouts, and a slew of them are no longer valid because of domain name changes.

There is no internal mechanism to remove URLs from this dialogue box, and I had not found anything on the Internet on how to do this. I was certain that this information had to be in a configuration file somewhere.

I ran across this page: Directories used by the IDE, which headed me in the correct direction. I eventually found the file I was looking for (using Mac OS X):

~/Library/Preferences/<PRODUCT><VERSION>/options/vcs.xml

This file has a set of <UrlAndUserName> elements that can be individually deleted, as needed:


<application>
  <component name="GitRememberedInputs">
    <option name="visitedUrls">
      <list>
        …
        <UrlAndUserName>
          <option name="url" value="https://github.com/appium/sample-code.git" />
          <option name="userName" value="" />
        </UrlAndUserName>
        …
      </list>
    </option>
    <option name="cloneParentDir" value="$USER_HOME$/IdeaProjects" />
  </component>
</application>
 
Exit IntelliJ IDEA completely, and start it up again. The next time you use the Clone Repository dialogue box (e.g., using "Check out from Version Control" in the Welcome dialogue box), you will see the list reduced to whatever entries you left in the vcs.xml file.

This solution was tested in version 2017.1, and I confirmed the same file location for version 2016.1.

Kubernetes Readiness and Liveness with Apache Kafka REST Proxy

When setting up Readiness and Liveness checks in Kubernetes for Kafka connectors, the use of the httpGet described in my previous blog post (Kubernetes Readiness and Liveness with Spring Boot Actuator) is not an option because there is no endpoint to reference. These can be deployed with the Apache Kafka REST Proxy, which gets us on the right path, but doesn't quite work how we want in this respect.

The Kafka REST Proxy provides endpoints that allow one to get some basic status info about connectors. However, the standard Kubernetes httpGet calls use status code >= 200 and < 400 to determine the status, and since the Kafka REST status endpoint always provides a 200 status code, it is not possible to use this methodology to determine if a connector is down.

What we would like to do is check the content of the status call, and do a string comparison. For example, when the service is up, the status endpoint indicates that the state is "RUNNING":

# curl http://10.30.128.1:8083/connectors/mysql-kafka-connector/status

{"name":"mysql-kafka-connector","connector":{"state":"RUNNING","worker_id":"10.30.128.1:8083"},"tasks":[{"state":"RUNNING","id":0,"worker_id":"10.30.128.1:8083"}]}

We can pause the connector using this endpoint:

# curl -i -X PUT http://10.30.128.1:8083/connectors/mysql-kafka-connector/pause

HTTP/1.1 202 Accepted

And then the state is changed to PAUSED:

# curl http://10.30.128.1:8083/connectors/mysql-kafka-connector/status

{"name":"mysql-kafka-connector","connector":{"state":"PAUSED","worker_id":"10.30.128.1:8083"},"tasks":[{"state":"PAUSED","id":0,"worker_id":"10.30.128.1:8083"}]}


To accomplish this check, we can leverage the exec command probe:

readinessProbe: 

  exec: 

    command:

      - /bin/sh 

      - -c 

      - curl -s http://127.0.0.1:8083/connectors/mysql-kafka-connector/status | grep "RUNNING"

  initialDelaySeconds: 240

  periodSeconds: 5

  timeoutSeconds: 5 

  successThreshold: 1

  failureThreshold: 10 

livenessProbe:

  exec: 

    command: 

      - /bin/sh

      - -c 

      - curl -s http://127.0.0.1:8083/connectors/mysql-kafka-connector/status | grep "RUNNING"

  initialDelaySeconds: 300 

  periodSeconds: 60

  timeoutSeconds: 10 

  successThreshold: 1

  failureThreshold: 3

The the exec command allows us to execute a shell command. In this case:

• running the shell (/bin/sh)
• telling it to run a single command (-c)
• with the command being a cURL call to the specific connector status endpoint, and grepping for the string "RUNNING"

When the grep is successful, Kubernetes interprets this as a success. If the grep comes back empty (i.e., "RUNNING" is not found), then it gets viewed as a failure. You can then test this on the server by pausing the service in question as described above.

To get the service running again and start passing readiness and liveness again, then you will want to use the RESUME endpoint.

# curl -i -X PUT http://10.30.128.1:8083/connectors/mysql-kafka-connector/pause

HTTP/1.1 202 Accepted

Kubernetes Readiness and Liveness with Spring Boot Actuator

In Kubernetes, "readiness" is the indicator that the service is ready to accept traffic, and is only performed at the beginning of a pod's life cycle. "Liveness" is a periodic health check that should indicate that the service is still functional within the pod.
More details can be found in the documentation at
https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/

Spring Boot is a stand-alone Spring environment well suited for micro services.
https://projects.spring.io/spring-boot/

Spring Boot has a bundle available called "Actuator" that exposes several helpful endpoints, one of which is a "health" endpoint. The simple version of this endpoint returns a simple JSON with a status UP or DOWN.
https://github.com/spring-projects/spring-boot/tree/master/spring-boot-actuator

One particular feature of the health endpoint that is useful is that besides the text indicator in the JSON response, it also signals up and down through the status code. UP = 200, and DOWN = 503 (Service Unavailable).

Putting this all together, the readiness and liveness configuration in the Kubernetes deployment YAML can look something like this:

readinessProbe:
httpGet:
  scheme: HTTP
  path: /health
  port: 8080
initialDelaySeconds: 240
periodSeconds: 5
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 10
livenessProbe:
httpGet:
  scheme: HTTP
  path: /health
  port: 8080
initialDelaySeconds: 300
periodSeconds: 60
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 3

In this example, we instruct Kubernetes to wait 240 seconds to allow the application to start before performing the check. It will retry up to 10 times with a five second pause between each try. It will wait for a maximum of 5 seconds for a result to be returned. After 10 failures, the pod will be restarted.

For the liveness check, we instruct Kubernetes to wait 300 seconds, and to check every 60 seconds. If three failures occur in a row, the pod will be restarted.

A toString Comparator for arbitrary objects

I recently ran into a problem where I wanted to add objects attained by reflection to a TreeSet. In this specific case, I had a problem when a Locale object was retrieved. It could not be added to the TreeSet because it is not "Comparable." It did, however, have toString values that could be sorted intelligibly.

I decided to create a Comparator that would handle arbitrary objects, and allow them to be added to a TreeSet of type Object:

The caveat is that this works best when the object in question overrides the default toString method in a meaningful way. If the base Object.toString() method is inherited by the object, then the sorting will most likely appear random, since the method looks like this:

    public String toString() {
        return getClass().getName() + "@" + Integer.toHexString(hashCode());
    }