Connecting to the GAE development server running on another machine

Google App Engine comes with a nice development server located in APP_ENGINE_HOME/bin/dev_appserver.sh

The syntax to run it is:  dev_appserver.sh PATH_OF_YOUR_GAE_EXPLODED_WAR

When you do that, your application is accessible from http://localhost:8080, however connecting to it from another machine using http://YOUR_IP:8080 will probably not work.

If this is your case, then it’s probably because it only listens on your loopback address (127.0.0.1)

[laurent@localhost ~]$ netstat -an | grep 8080
tcp 0 0 ::ffff:127.0.0.1:8080 :::* LISTE

To fix that, launch your dev appserver with “-a 0.0.0.0″ and you should be good to go: dev_appserver.sh -a 0.0.0.0 PATH_OF_YOUR_GAE_EXPLODED_WAR

[laurent@localhost ~]$ netstat -an | grep 8080
tcp        0      0 :::8080                     :::*                        LISTEN

… and if it still doesn’t work, check your firewall settings: maybe the access to post 8080 is blocked.

Laurent KUBASKI

Classes in Javascript

Instance properties, instance methods, class properties, class methods: yes, you can have all of these in Javascript, but you need to remember when to use ‘var’, when to use ‘this’ and when to use ‘that’.

Since I don’t seem to be able to easily remember all this stuff I wrote a small code sample that uses all of them. To run it, open the Javascript console in Chrome (CTRL-SHIFT-J on Windows) and run this baby:


function MyConstructor() {
  var that = this;
  this.publicInstanceProperty = "publicInstanceProperty";
  var privateInstanceProperty = "privateInstanceProperty";
  var privateInstanceMethod = function() {
    console.assert(that.publicInstanceProperty === "publicInstanceProperty");
    console.assert(privateInstanceProperty === "privateInstanceProperty");
  }
  privateInstanceMethod();
  // A privileged method is able to access the private variables and methods
  // it is itself accessible to the public methods and the outside.
  this.privilegedInstanceMethod = function() {
    console.assert(that.publicInstanceProperty === "publicInstanceProperty");
    console.assert(privateInstanceProperty === "privateInstanceProperty");
  }
};
MyConstructor.prototype.publicInstanceMethod = function() {
  console.assert(this.publicInstanceProperty === "publicInstanceProperty");
  console.assert(this.privateInstanceProperty === undefined);
  // this.privateInstanceMethod(); // TypeError: Object #<MyConstructor> has no method 'privateInstanceMethod'
};
MyConstructor.classProperty = "classProperty";
MyConstructor.classMethod = function() { };

//////////
// main //
//////////
var ctx = new MyConstructor();
console.assert(ctx.publicInstanceProperty === "publicInstanceProperty");
console.assert(ctx.privateInstanceProperty === undefined);
ctx.publicInstanceMethod();
// ctx.privateInstanceMethod(); // TypeError: Object #<MyConstructor> has no method 'privateInstanceMethod'
ctx.privilegedInstanceMethod();
console.assert(MyConstructor.classProperty === "classProperty");
MyConstructor.classMethod();

Douglas Crockford was the first to write about creating private instance properties/methods in Javascript. Each time you see the “var that = this” trick, that’s him !

However, I find this stackoverflow post to be easier to understand.

Navigating in a Word 2010 document using hyperlinks and bookmarks

OK, I’ve clicked on an hyperlink in a Word 2010 document and I’m now in a totally different page: “how the hell do I jump back to the page I was before clicking on the link ?“.

As surprising as it seem, there is no “back” button anywhere.

Took me at least 15 minutes to find the answer, which is: hold ALT+left arrow key.

Happy now ? :-)

Laurent KUBASKI

 

How Tomcat and Jetty generate the sessionId

Ever wondered how Tomcat and Jetty generate a unique sessionId ? (I’m talking about the one returned by HttpSession.getId()).

Here is how is works:

Tomcat 7.0.35

For Tomcat, the whole logic is in SessionIdGenerator.generateSessionId()


public class SessionIdGenerator {

/**
 * Generate and return a new session identifier.
 */
public String generateSessionId() {
byte random[] = new byte[16];
// Render the result as a String of hexadecimal digits
StringBuilder buffer = new StringBuilder();
int resultLenBytes = 0;

while (resultLenBytes < sessionIdLength) {
getRandomBytes(random);
for (int j = 0; j < random.length && resultLenBytes < sessionIdLength; j++) {
byte b1 = (byte) ((random[j] & 0xf0) >> 4);
byte b2 = (byte) (random[j] & 0x0f);

if (b1 < 10) buffer.append((char) ('0' + b1));
else buffer.append((char) ('A' + (b1 - 10)));

if (b2 < 10) buffer.append((char) ('0' + b2));
else buffer.append((char) ('A' + (b2 - 10)));

resultLenBytes++;
 }
 }

if (jvmRoute != null && jvmRoute.length() > 0) {
 buffer.append('.').append(jvmRoute);
 }

return buffer.toString();
 }

/**
 *
 */
 private void getRandomBytes(byte bytes[]) {
SecureRandom random = randoms.poll();
 if (random == null) {
 random = createSecureRandom();
 }
 random.nextBytes(bytes);
 randoms.add(random);
 }

 /**
 * Create a new random number generator instance we should use for
 * generating session identifiers.
 */
 private SecureRandom createSecureRandom() {

SecureRandom result = null;

long t1 = System.currentTimeMillis();
 if (secureRandomClass != null) {
 try {
 // Construct and seed a new random number generator
 Class<?> clazz = Class.forName(secureRandomClass);
 result = (SecureRandom) clazz.newInstance();
 } catch (Exception e) {
 log.error(sm.getString("sessionIdGenerator.random",
 secureRandomClass), e);
 }
 }

if (result == null) {
 // No secureRandomClass or creation failed. Use SecureRandom.
 try {
 if (secureRandomProvider != null &&
 secureRandomProvider.length() > 0) {
 result = SecureRandom.getInstance(secureRandomAlgorithm,
 secureRandomProvider);
 } else if (secureRandomAlgorithm != null &&
 secureRandomAlgorithm.length() > 0) {
 result = SecureRandom.getInstance(secureRandomAlgorithm);
 }
 } catch (NoSuchAlgorithmException e) {
 log.error(sm.getString("sessionIdGenerator.randomAlgorithm",
 secureRandomAlgorithm), e);
 } catch (NoSuchProviderException e) {
 log.error(sm.getString("sessionIdGenerator.randomProvider",
 secureRandomProvider), e);
 }
 }

if (result == null) {
 // Invalid provider / algorithm
 try {
 result = SecureRandom.getInstance("SHA1PRNG");
 } catch (NoSuchAlgorithmException e) {
 log.error(sm.getString("sessionIdGenerator.randomAlgorithm",
 secureRandomAlgorithm), e);
 }
 }

if (result == null) {
 // Nothing works - use platform default
 result = new SecureRandom();
 }

// Force seeding to take place
 result.nextInt();

long t2 = System.currentTimeMillis();
 if ((t2 - t1) > 100)
 log.info(sm.getString("sessionIdGenerator.createRandom",
 result.getAlgorithm(), Long.valueOf(t2 - t1)));
 return result;
 }

}

Jetty 8.1.9

For Jetty, the logic is in  AbstractSessionIdManager.newSessionId()

public abstract class AbstractSessionIdManager extends AbstractLifeCycle implements SessionIdManager {
/**
 * Set up a random number generator for the sessionids.
 * <p/>
 * By preference, use a SecureRandom but allow to be injected.
 */
 public void initRandom() {
 if (_random == null) {
 try {
 _random = new SecureRandom();
 } catch (Exception e) {
 LOG.warn("Could not generate SecureRandom for session-id randomness", e);
 _random = new Random();
 _weakRandom = true;
 }
 } else
 _random.setSeed(_random.nextLong() ^ System.currentTimeMillis() ^ hashCode() ^ Runtime.getRuntime().freeMemory());
 }

/**
 * Create a new session id if necessary.
 *
 * @see org.eclipse.jetty.server.SessionIdManager#newSessionId(javax.servlet.http.HttpServletRequest, long)
 */
 public String newSessionId(HttpServletRequest request, long created) {
 synchronized (this) {
 if (request != null) {
 // A requested session ID can only be used if it is in use already.
 String requested_id = request.getRequestedSessionId();
 if (requested_id != null) {
 String cluster_id = getClusterId(requested_id);
 if (idInUse(cluster_id))
 return cluster_id;
 }

// Else reuse any new session ID already defined for this request.
 String new_id = (String) request.getAttribute(__NEW_SESSION_ID);
 if (new_id != null && idInUse(new_id))
 return new_id;
 }

// pick a new unique ID!
 String id = null;
 while (id == null || id.length() == 0 || idInUse(id)) {
 long r0 = _weakRandom
 ? (hashCode() ^ Runtime.getRuntime().freeMemory() ^ _random.nextInt() ^ (((long) request.hashCode()) << 32))
 : _random.nextLong();
 if (r0 < 0)
 r0 = -r0;
 long r1 = _weakRandom
 ? (hashCode() ^ Runtime.getRuntime().freeMemory() ^ _random.nextInt() ^ (((long) request.hashCode()) << 32))
 : _random.nextLong();
 if (r1 < 0)
 r1 = -r1;
 id = Long.toString(r0, 36) + Long.toString(r1, 36);

//add in the id of the node to ensure unique id across cluster
 //NOTE this is different to the node suffix which denotes which node the request was received on
 if (_workerName != null)
 id = _workerName + id;
 }

request.setAttribute(__NEW_SESSION_ID, id);
 return id;
 }
 }

}

Laurent KUBASKI

Ordering wordpress menu items

Note: when I talk about wordpress here, I’m talking about the http://www.wordpress.com online service.

There are many available themes available for your wordpress blog, and they all come with at least one menu. The one I’m using is called “Enterprise” and comes with two menus. By default, the top menu contains my blog “pages” and the bottom menu contains my blog “categories”:

blog_menu

A common requirement is: how do I change the ordering of the menu entries ? (ie: what if I want the “uncategorized” category to be the last one ?)

Some people suggested adding a number if front of each category (like “1. Android” and “2. Linux”), hoping that wordpress would display the entries in the ascending order, but this doesn’t work. The truth is that this is not possible if you use the default theme menus: you need to create your own.

To do this, go to the “Appearance > Menu” section of your dashboard:

dashboard_menu

You will then see a “Theme locations” section with two dropdown lists. This is where you can specify your top and bottom menus (that is: if your theme allows two menus of course). By default, nothing is selected, which means that the theme built-in menus are used. As I said above, if you want to control exactly how your menus are organized, you need to create your own menus instead of using the built-in ones:

theme_locations_default

To create a new menu, click on the “+” button:

create_menu

Then, select the pages and/or categories that should be part of your menu and click on the “Add to menu” button:

pages_categories

Once the entries are added to your menu, use drag and drop to move them around until they are in the correct order.

drag_category

Finally, don’t forget to click on the “Save Menu” button !

Then, go back to the “Theme Locations” section and set your newly created menu as the primary one:

theme_locations_custom

There you go:

blog_menu_custom

Rotating photos from mobile devices uploaded to Google App Engine

I’m currently working on a Google App Engine application that allows users to upload photos from their mobile device and then view these photos in a slideshow.

So far, so good, until I realized that some of the uploaded images where sometimes rotated.

What the… ?

This is where I learnt about the concept of EXIF metadata. Long story short: each photo on your mobile device is stored as a jpeg file which contains the photo itself, but also a bunch of extra metadata, like the photo orientation.

This photo orientation depends on your mobile device orientation when the photo is taken. Using the iphone as an example, you basically have 4 possible positions:

  • If the iphone “home” button points downwards when the photo is taken, the photo is rotated 90° counter clockwise (“turned to the left” if you prefer)
  • If the iphone “home” button points upwards when the photo is taken, the photo is rotated 90° clockwise (“turned to the right” if you prefer)
  • If the iphone “home” button points to the left when the photo is taken,  the photo is rotated 180°.
  • If the iphone “home” button points to the right when the photo is taken,  the photo is not rotated (I guess this is how the iphone is supposed to be held when taking pictures).

What I ended up doing is:

  1. User uploads a photo
  2. GAE application determines what the photo orientation is and rotates it accordingly
  3. GAE application stores the rotated photo in the datastore

How to determine the photo orientation

As previously explained, you need to access the photo EXIF metadata and get the value of the “orientation” property.

Luckily, there is a great open-source project that can help us extract this value: it’s called metadata-extractor.

Here is how I used this library:


private int getEXIFOrientation(byte[] bytes) {
int orientation = -1; // default = unknown
 ByteArrayInputStream bis = null;
 try {
 bis = new ByteArrayInputStream(bytes);
 Metadata metadata = ImageMetadataReader.readMetadata(new BufferedInputStream(bis), false);
 ExifIFD0Directory exifDir = metadata.getDirectory(ExifIFD0Directory.class);
 if (exifDir != null) {
 orientation = exifDir.getInt(274); // 274 is the EXIF orientation standard code
 }
 } catch (Exception e) {
 log.warning("Couldn't extract EXIF orientation from image");
 } finally {
 if (bis != null) try {
 bis.close();
 } catch (IOException e) {
 // nothing
 }
 }
 return orientation;
}

How do I know that 274 is the internal code for the orientation EXIF tag ? Because of this link.

The orientation property can have 8 different values, but only 4 of them (in bold) are used:

  • 1: image is Normal-> that’s the orientation value you get when the iphone home button is on the right
  • 2: image is flipped horizontally
  • 3: image is rotated 180° -> that’s the orientation value you get when the iphone home button is on the left
  • 4: image is flipped vertically
  • 5: image is rotated 90° CCW and flipped vertically
  • 6: image is rotated 90° CCW -> that’s the orientation value you get when the iphone home button is upwards
  • 7: image is rotated 90° CW and flipped vertically
  • 8: image is rotated 90° CW -> that’s the orientation value you get when the iphone home button is downwards

How to rotate the image

That’s the easy part: we just use the GAE Image service, more precisely: ImagesServiceFactory.makeRotate().

  • orientation = 1 -> don’t rotate
  • orientation = 3 -> rotate 180° clockwise
  • orientation = 6 -> rotate 90° clockwise
  • orientation = 8 -> rotate -90° clockwise (or 90° counter clockwise if you prefer).

So we end up with:


private byte[] rotateImage(byte[] bytes) {
byte[] result = bytes;
int orientation = getEXIFOrientation(bytes);
int degrees = -1;
if (orientation == 3) degrees = 180;
else if (orientation == 6) degrees = 90;
else if (orientation == 8) degrees = -90;
if (degrees != -1) {
Image img = ImagesServiceFactory.makeImage(bytes);
Transform rotation = ImagesServiceFactory.makeRotate(degrees);
// GAE changes output format from JPEG to PNG while rotating the image if the expected output format is not given:
OutputSettings settings = new OutputSettings(ImagesService.OutputEncoding.JPEG);
settings.setQuality(1);
img = ImagesServiceFactory.getImagesService().applyTransform(rotation, img, settings);
result = img.getImageData();
}
return result;
}

Laurent KUBASKI

Serving dynamic images in Google App Engine

Serving a static image in Google App Engine is as easy as:


<img src="/images/01.jpg">

But what if the image is stored in the datastore as a BLOB ?

There is an official “Serving Dynamic Images with Google App Engine” article in the Google App Engine knowledge base, but it doesn’t explain all the needed steps to accomplish this… and that’s exactly what I’m going to do.

The thing is: the “src” attribute of the HTML “img” tag doesn’t need to be the path of an image on the server filesystem. It can also be the url of a Servlet that will return an image.

So, assuming that you know the id of your image in the datastore, you can have this:


<img src="/serveImage?id=<%=key.getId()%>">

/serveImage is the URI of my ServeImageServlet, as configured in web.xml:


<servlet>
<servlet-name>ServeImageServlet</servlet-name>
<servlet-class>fr.lk.servlet.ServeImageServlet</servlet-class>
</servlet>

<servlet-mapping>
<servlet-name>ServeImageServlet</servlet-name>
<url-pattern>/serveImage</url-pattern>
</servlet-mapping>

Now let’s see ServeImageServlet.java:

public class ServeImageServlet extends HttpServlet {

@Override
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
String id = req.getParameter("id");
byte[] bytes = new DAO().getImage(Long.parseLong(id));
resp.setContentType("image/jpeg");
resp.getOutputStream().write(bytes);
}
}

As you can see, the “id” request parameter is used to retrieve the image from the datastore (as a raw byte array), and we just directly write it to the response output stream.

Laurent KUBASKI

Follow

Get every new post delivered to your Inbox.