9 Feb 2017

Fixing Firefox issues with column headers for dojox.grid.EnhancedGrid

In Firefox, clicking a dojox.grid.EnhancedGrid’s column header to change sorting only works for the first column. Clicking another column’s header only shows a splitter. The same goes for right-clicking a column header to bring up a popup menu: it only works in the first column. In other browsers this works fine.

The cause seems to be a workaround for an issue in older Firefox versions, implemented in EnhancedGrid.js. The issue no longer exists in recent Firefox versions, but the workaround introduces the issues mentioned in the first paragraph. This is detailed in this bug. The bug also has a proposed fix, which never got into the dojo release (support for EnhancedGrid is very limited; dojo urges us to migrate to dgrid and/or gridx instead).

So we need to apply our own workaround to get the grid working properly again in Firefox (or migrate to dgrid/gridx if we have the time and the energy). A workaround to work around the workaround, if you see what I mean. This means neutralizing the function createView in EnhancedGrid.js. I managed to do so by assigning my own createView function to the grid which just calls this function in the base class (DataGrid). I added the following to the grid's data-dojo-props:


createView: function () { return dojox.grid.DataGrid.prototype.createView.apply(this, arguments); }

26 Feb 2014

Implementing an Android radio button dialog in QPython 3

QPython 3 is a neat app that allows you to run Python 3 scripts on an Android device. All my scripts that I originally created on Linux worked without problems. They were all console scripts though, and I decided to Androidify them a bit. Instead of asking multiple choice questions and making the user type in the answers in the console, the scripts should show nice radio button dialogs:

This was not as easy to accomplish as I had hoped. There's not a lot of documentation yet, it's scattered over various websites, and much of it targets Python 2.7. But eventually I got it to work, like this:


import androidhelper

def radioButtonDialog(droid, question, options):
    droid.dialogCreateAlert(question)
    droid.dialogSetSingleChoiceItems(options)
    droid.dialogSetPositiveButtonText("OK")
    droid.dialogSetNegativeButtonText("Cancel")
    droid.dialogShow()
    response = droid.dialogGetResponse().result
    if "which" in response:
        result = response["which"]
    if result == "negative":
        raise Exception("Aborted")
    selectedItems = droid.dialogGetSelectedItems().result
    droid.dialogDismiss()
    return selectedItems.pop()

droid = androidhelper.Android()
options = ["I agree", "I'm not sure", "I don't agree"]
try:
    result = radioButtonDialog(droid, "The penguin is awesome.", options)
    message = "You chose: " + options[result]
except Exception:
    message = "Oh, you don't want to tell me. That's ok."
droid.makeToast(message)

20 Feb 2014

Creating a baby journal with TiddlyWiki 5

Keeping a journal about your baby is fun. It allows you to keep track of those events in your baby's life and development that are most important to you, and to tell them in your own words. Sometimes it's not just fun but rather a necessity to keep a baby journal, like when the doctor wants you to keep a record of how many nappies the little one uses daily and what's in them.

There are plenty of online services that can help to keep such a diary, but you need to have an internet connection to access them, and you have to trust them to keep your data safe.

This blog post descibes how to keep a journal that will also be available to you when you're offline. It will be just one file that you can store anywhere and view or edit with any modern web browser. Journal entries can contain rich text and images, and automatically show the entry's date and your baby's age at that date:

TiddlyWiki

The journal will be based on TiddlyWiki 5, created by Jeremy Ruston. Now, this is a blog mainly about developing software, and here's where things start to get all technical and fun. However, if you're a non-technical parent who just came here looking for a simple baby journal application, you may (almost) stop reading, download the finished version of the file we're about to create over here, open it in your favourite browser, enter your baby's details in the Personal data section and start creating journal entries. I think it all works pretty straightforward, but do remember to click the big Save button before exiting your browser. Check the TiddlyWiki website for any additional help.

Still reading? Excellent. So I mentioned TiddlyWiki. It's an interesting piece of software that implements a wiki type of application in a single html file which contains all the html, JavaScript and css needed to run the application as well as the data that the user puts into the wiki. It's highly customizable and has many different uses. For our baby journalling purpose, we will create a faily standard TiddlyWiki file with one customization: the entry's date and the baby's age will appear at the top of each journal entry.

Getting started

To start creating the journal, you need an empty TiddlyWiki. Download it at http://www.tiddlywiki.com by clicking the large green button that says Download Empty. Save the file wherever you like, and then open it with your browser. Let's start by giving your journal a title. Click the Control Panel button and enter a title at Title of this TiddlyWiki, and maybe a subtitle at Subtitle. Then click the Save button (see above). Your browser will ask you for a location to save the file, which can be the same location where you originally saved the file.

There, that was the basic process of editing the TiddlyWiki file: open it in the browser, make some modifications and then click Save. Of course, the saving process is a bit awkward with that file requestor. For Firefox, it can be made a one-click affair by installing an extension. You could also use TiddlyDesktop which is a standalone application providing a browser and easy management of TiddlyWiki files. See the TiddlyWiki website for more information.

The first tiddler

In TiddlyWiki, information is stored in tiddlers. They are comparable to pages or articles in other wikis. Let's create a tiddler that contains some general information about your baby. Click on the big + button and enter as title for the new tidder: Personal data. In the text area below the title you can write whatever you like about the baby, but there are 2 special bits of information that we want to use in other parts of the journal too: the baby's first name and birth date. To easily access these from elsewhere, we put them in fields. At Add a new field, enter

firstname
and click the Add button. Now you can enter a value for the field, so enter your baby's first name. Repeat the process for the new field
birthdate
and enter the baby's birth date in the notation yyyymmdd (for example if the baby was born on 31 October 2013, enter 20131031). Finish the creation of the new tiddler by clicking ✓. This stores the new tiddler in memory, but not yet on disk. To also save it on disk, click the large Save button again (see above).

Now you can use the values of these fields wherever you want with syntax:

{{Personal data!!firstname}}
and
{{Personal data!!birthdate}}
For example, as title (in Control Panel, see above) you could set:
{{Personal data!!firstname}}'s journal
Then your baby's name would appear in the title. You'll notice that after saving, the Personal data tiddler doesn't show these 2 fields and their values anymore; they're only visible in edit mode, not in view mode. To also see them in view mode, the text of the tiddler should explicitly reference them. Click on the tiddler's pen icon to modify it and write the following wikitext into the tiddler's text area, which will show the field names and values in a table:

|Birth date (yyyymmdd): |{{!!birthdate}}|
|First name: |{{!!firstname}}|
Here the tiddler's name can be omitted before the !!, because it's the current tiddler itself. By the way, the tiddler's text is actually also a field of the tiddler. It's a standard field that any tiddler has, whereas the 2 new fields you just added are custom fields that only the Personal data tiddler has.

Importing a tiddler from another TiddlyWiki

As said in the introduction, we want to show the date of a journal entry in its header. Unfortunately TiddlyWiki has no built-in support for showing dates. It does support macros to add custom functionality though, so let's make a JavaScript macro to do it. Actually someone already made such a macro, so you just need to import it into your journal. To do this, open a new browser window and in it, go to Stephan Hradek's site http://tiddlystuff.tiddlyspot.com and click on the link to the tiddler "dateTimeMacro". At the bottom of this tiddler, there is a link called $:/macros/skeeve/dateTime.js. Drag and drop this link into your journal (from one window to the other). Now you have a macro for showing the date of a tiddler. Have a look at it to get a general idea about what JavaScript macros look like in TiddlyWiki. (If you can't see it anymore: it should be in the More -> System section of your Control Panel.)

Creating your own JavaScript macro

Before we go on using the dateTimeMacro to create the journal entry header, we need one more macro there: to calculate and show the age of your baby. This macro you will have to make by yourself. Create a new tiddler and call it $:/macros/penguin/age.js. Set its Type field to "application/javascript". Add a field called "module-type" and set it to "macro". Then, in the tiddler text paste the following JavaScript code that implements the macro:

(function(){

"use strict";

exports.name = "age";

exports.params = [ { name: "birthDate" }, { name: "currentDate" } ];

exports.run = function(birthDate, currentDate) {
  var getNormalizedDate = function (date, referenceDate) {
    var numberOfDaysInMonth = (new Date(date.getFullYear(), date.getMonth()+1, 0)).getDate();
    if (numberOfDaysInMonth < referenceDate.getDate()) {
      return numberOfDaysInMonth;
    }
    else {
      return referenceDate.getDate();
    }
  }
  var getAgeString = function (birthDate, currentDate) {
    var tmpBirthDate = new Date(birthDate);
    var tmpCurrentDate = new Date(currentDate.getFullYear(), currentDate.getMonth(), currentDate.getDate());
    if (tmpCurrentDate < tmpBirthDate) return "not yet born";
    tmpCurrentDate.setHours(12);
    var ageDays = 0;
    while (tmpCurrentDate.getDate() != getNormalizedDate(tmpCurrentDate, tmpBirthDate)) {
      ageDays++;
      tmpCurrentDate.setDate(tmpCurrentDate.getDate() - 1);
    }
    tmpCurrentDate.setDate(1);
    tmpBirthDate.setDate(1);
    var ageMonths = 0;
    while (tmpCurrentDate.getMonth() != tmpBirthDate.getMonth()) {
      ageMonths++;
      tmpCurrentDate.setMonth(tmpCurrentDate.getMonth() - 1);
    }
    var ageYears=0;
    while (tmpCurrentDate.getFullYear() != tmpBirthDate.getFullYear()) {
      ageYears++;
      tmpCurrentDate.setFullYear(tmpCurrentDate.getFullYear() - 1);
    }
    return ageYears + ' year' + (ageYears == 1 ? '' : 's') + ', ' + ageMonths + ' month' + (ageMonths == 1 ? '' : 's') + ' and ' + ageDays + ' day' + (ageDays == 1 ? '' : 's');
  };

  if (typeof(currentDate) == "undefined" || currentDate == "" || isNaN(currentDate)) {
    currentDate = new Date();
  }
  else
  {
    currentDate = new Date(currentDate.substring(0,4), currentDate.substring(4,6) - 1, currentDate.substring(6, 8));
  }
  if (typeof(birthDate) == "undefined" || birthDate == "" || isNaN(birthDate)) {
    birthDate = new Date();
  }
  else
  { 
    birthDate = new Date(birthDate.substring(0,4), birthDate.substring(4,6) - 1, birthDate.substring(6, 8));
  }
  return getAgeString(birthDate, currentDate);
};
})();
For any JavaScript macro in TiddlyWiki, the exports object needs 3 attributes: name for the macro name (age), params for its parameters (birthDate and currentDate) and run for the function to run when it's invoked.

Defining the header for journal entries

Now you can create the header that will appear at the top of all journal entries. Create a new tiddler called JournalEntryHeader and paste the following text in it:

<$list filter="[is[current]tag[JournalEntry]]">
//<$macrocall $name="dateTime" ts={{!!created}} format="DDD DD MMM YYYY"/>//<br>
{{Personal data!!firstname}} is <$macrocall $name="age" birthDate={{Personal data!!birthdate}} currentDate={{!!created}} /> old.<hr></$list>
This looks rather complicated, so let's see what's happening there.

The whole header is enclosed in a $list construct. This list has 0 or 1 elements, depending on the tiddler in which it appears. It's a rather awkward construction, but it basically says: show information about the current tiddler, but only if this tiddler has tag JournalEntry. This way, we can include the header in all tiddlers (which is easiest to do, see below), while it only shows text for some tiddlers.

Inside the list, our 2 macros are used. First there's the dateTime macro which will show the tiddler's creation date (the field created) in the format "name_of_weekday monthday name_of_month year". It's enclosed by // to make it appear in italics. Then there's a line that shows the baby's age. It references the baby's name set in Personal data and then calls the age macro which computes the age at the tiddler's creation date, given the birth date defined in Personal data.

To include the header in all tiddlers, you just need to add a tag $:/tags/ViewTemplate to the JournalEntryHeader tiddler. However, that doesn't include it at the top of the tiddler, but at the bottom. It's kind of expected for a header to appear at the top, so let's make sure that happens. For this, open the tiddler $:/tags/ViewTemplate (to be found in Control Panel -> More -> Shadows) and edit it. In its field "list", insert JournalEntryHeader before $:/core/ui/ViewTemplate/body. Now the header will appear before the tiddler's body.

Now you can test the macros. It's important to note that a new macro doesn't work until you save the tiddler and then refresh the browser, so first do that. Then create a new tiddler and add a tag JournalEntry to it. Type some text and a title, and then save the tiddler. The new tiddler should now show the header with the date (today) and the baby's age. If not, you may have made a typo in one of the macros or elsewhere. Maybe you noticed that the header was not shown while you were editing the tiddler. To also show the header in edit mode, you need to do with $:/tags/EditTemplate what you did with $:/tags/ViewTemplate.

Putting things together: the main page

Now it's time to bring everything together and finish the journal. Let's create a tiddler entitled Main page and make sure it's always visible when we start the journal. To ensure that, go to Control Panel and put [[Main page]] in the Default Tiddlers.

The main page's header

At the top of the main page, let's put information similar to what's in the headers of journal entries, but now related to the current date. So when you open the journal, you'll always see the current date and your baby's current age. This is accomplished by calling the macros without the currentDate argument (which makes it default to now). Put the following at the top of your Main page tiddler:


//<<dateTime format: "DDD DD MMM YYYY">>//<br>
Today {{Personal data!!firstname}} is <$macrocall $name="age" birthDate={{Personal data!!birthdate}} /> old.<hr>
Note the much shorter syntax for the dateTime macrocall: <<.>> instead of <$macrocall>..</$macrocall>. This shorter syntax can't always be parsed correctly, which is why elsewhere we use the longer one.

A button to create new journal entries

Creating a new journal entry is still pretty awkward: first you have to click the button to create a new tiddler, then you need to assign a JournalEntry tag to it. Luckily we can do both in one click by adding a custom "Create journal entry" button. This button will create a new tiddler based on a template (a "Skeleton" in TiddlyWiki terminology). In this template, then, you can specify what stuff you want to be in every journal entry. This is at least the tag JournalEntry, and maybe some fields for specific data you want to keep track of, such as weight, height, number of full nappies... So create a new tiddler called JournalEntrySkeleton and add the tag JournalEntry to it plus any other stuff you want to be in each journal entry, and then add the following to the Main page to make the button appear there:

<$button message="tm-new-tiddler" param="JournalEntrySkeleton">New journal entry</$button>

A table of contents

Finally, it might be useful to have a list with links to all journal entries you created, plus a link to the Personal data tiddler. Add the following to the Main page tiddler:


[[Personal data]]
<hr>
<h2>Journal entries:</h2>

<$list filter="[tag[JournalEntry]!title[JournalEntrySkeleton]sort[created]]"><$link to={{!!title}}><$view field="title" /> (<$macrocall $name="dateTime" ts={{!!created}} format="DD MMM YYYY"/>)<br></$link>

Here the $list construct is used again, this time to actually produce a list (as opposed to just those 0 or 1 lines in the JournalEntryHeader). It lists all tiddlers with tag journalEntry except the skeleton, sorted by creation date.

And there you have it, a baby journal made with TiddlyWiki 5. I can't guarantee that your baby will be interested to read it when he/she's older, but it will surely be a fun document for you and your partner to look back on those exciting but often exhausting baby years.

Links

22 Jan 2014

Syntax highlighting for source code on Blogger 2: highlight.js

After using the excellent SyntaxHighlighter by Alex Gorbatchev for a while (see this post), I decided it was time for a change, and tried highlight.js. Here's how to set it up to work with a Blogger blog:
  1.  On your blog's dashboard, click Template and then Edit HTML. You'll get a warning that says that you may mess up your blog, but since you're an ace programmer anyway, you don't mind much and you click Proceed.
  2. Find the closing </head> tag and right above it, paste the following:
    
    <!--SYNTAX HIGHLIGHTER BEGINS-->
    <link href='http://yandex.st/highlightjs/8.0/styles/default.min.css' rel='stylesheet'/>
    <script src='http://yandex.st/highlightjs/8.0/highlight.min.js'></script>
    <script>hljs.initHighlightingOnLoad();</script>
    <!--SYNTAX HIGHLIGHTER ENDS-->
    
  3. Click Save.
  4. Now, to use the syntax hightlighting in your Blogger post, click HTML in the Edit Post window, and write code like this (the example is for HTML code - notice the &lt; instead of < to prevent issues):
    
    <pre><code>
    &lt;input type="Submit" value="Submit" />
    </code></pre>
    
    to get this in the resulting post:
    
    <input type="Submit" value="Submit" />
    

That's all!

Update: to get the horizontal scrollbar if the lines are too long to fit, add the following after the <link> line above:


<style type="text/css">
.hljs {
  overflow-x: auto;
}
</style>

20 Jan 2014

Getting a date's week number in JavaScript

JavaScript's Date object doesn't seem to have a built-in function to get the week number of a date. There are many JS libraries that do have support for this, but I needed to get the week number in an environment where including libraries is awkward (in a TiddlyWiki 5 macro, if you really want to know). So I decided to make my own function.

A search on StackOverflow revealed plenty of ways to do this, but they all seemed to use date subtractions based on the fact that a date is represented internally as the number of milliseconds since 1 January 1970. To me that fact feels like an internal implementation detail, so I don't really like to use it. Also, I don't (want to) know how things like leap seconds would influence the reliability of these subtractions. No choice then but to reinvent the wheel and come up with my own algorithm and implementation.

The ISO 8601 standard says that week 1 of a year is the first week with a Thursday in that year. Based on that, I came up with the following "algorithm":

  1. Given a date d, adapt this date to Thursday of the same week.
  2. Subtract 1 week from this date and repeat this until the result is in the previous year.
  3. The week number for date d is the number of subtractions needed in the previous step to get to the previous year.
This surely isn't the fastest way to do it, but it takes a maximum of 53 subtractions, so it does have constant time complexity. Note that it does use subtractions, so we might end up using those milliseconds again, except that subtracting days can also be done by using the fact that JavaScript's setDate function has a kind of "underflow" protection: if 0 or a negative value is passed, it adapts the date to the correct month. So we can delegate the subtractions to an internal JS function and assume that it works well.

All these considerations led me to the following JavaScript function. Please let me know if there are any flaws anywhere in the function, or in the considerations.


function getWeekNumber(date) {
  var addDays = 0, dayOfWeek = date.getDay(), 
    modifiedDate = new Date(date);
  // move to Thursday in same week as date
  if (dayOfWeek == 0) {
    addDays = -3;
  }
  else {
    addDays = 4 - dayOfWeek;
  }
  modifiedDate.setDate(modifiedDate.getDate() + addDays);
   
  // count weeks going back in time one week at a time
  // until we reach previous year
  var year = modifiedDate.getFullYear(), weekCount = 0;
  do {
    modifiedDate.setDate(modifiedDate.getDate() - 7);
    weekCount++;
  } while (modifiedDate.getFullYear() == year);
  return weekCount;
}

Note: the code assumes weeks start on Mondays. In some countries (and in JavaScript) weeks start on Sundays, so there the first if statement would not be needed.

Since we're on a web page, let's test the function:

Today we are in week

17 Jan 2014

Mounting a HiDrive in Ubuntu

Accessing a (free) Strato HiDrive as a regular filesystem in Ubuntu is possible, though not completely trivial to set up. Here is what I had to do to get it running:
  1. I chose the WebDAV protocol to access my HiDrive (another option would be sftp, but only if you have a paid account). For this, some packages needed to be installed:
    
    sudo apt-get install davfs2
    sudo apt-get install ca-certificates
    
  2. I wanted to mount and use the HiDrive as a normal Ubuntu user, not as root. For this, the davfs2 package needed some tuning. I performed the following command:
    
    sudo dpkg-reconfigure davfs2
    
    and I answered "Yes" when it asked "Should unprivileged users be allowed to mount WebDAV resources?".
  3. Also, I needed to add myself (my Ubuntu username being penguin) to the davfs2 group:
    
    sudo usermod -a -G davfs2 penguin
    
  4. I created the mountpoint and made sure it would be accessible for a regular Ubuntu user:
    
    sudo mkdir /hidrive
    sudo chmod 777 /hidrive
    
  5. To configure the mountpoint, I edited /etc/fstab:
    
    sudo vi /etc/fstab
    
    and added the following entry:
    
    https://webdav.hidrive.strato.com/  /hidrive  davfs  user,noauto  0  0
    
For most Ubuntu users, this would be enough and the HiDrive could be mounted by this command (notice the absence of "sudo"):

mount /hidrive
However, this gave me an error that said:
/sbin/mount.davfs: / is the home directory of user abc.
You can't mount into another users home directory
A web search showed that some people got a similar message, only for user "kernoops". This could then be solved by adding kernoops to the ignore_home entry in the file /etc/davfs2/davfs2.conf. "kernoops" seems to be present already in most Ubuntus, but this abc user was not. I think in my case there was some kind of user/group id conflict caused by the YP/NIS server where this abc user was defined. Anyway, adding also abc to that line solved my problem:

sudo vi /etc/davfs2/davfs2.conf
ignore_home       kernoops,distccd,abc

8 Feb 2012

Running interactive programs uninteractively in Bash scripts

Bash scripts are great for automating tasks. Some challenges arise though when you want to automate a process that involves programs that ask for user input by keyboard. You know, the kind of program that keeps asking things like: "Are you sure you want to do this (y/n)?". Most programs accept arguments that enable some kind of "non-interactive mode" for that. For example,


rm -i somefile
asks for confirmation before deleting somefile, while

rm -f somefile
does not. This of course makes the latter a lot more suitable for use in automated scripts. Unfortunately, not all programs provide these kind of arguments. What to do with them?

One option is to use redirection and echo the input into them, like so:


echo "yes" | rm -i somefile
or, if you want to remove a lot of files (and for some reason don't want to replace the -i argument with -f), with the brilliant command yes:

yes | rm -i somefile*

Things start to look ugly though when you have to provide a lot of different input texts. Say you have a program generate_random_files which, well, generates random files after asking in sequence:

  1. How many files?
  2. Size of files?
  3. Are you sure (y/n)?
Suppose our answer to the first question is 2, to the second question 4, and the third one y. Providing the input could be accomplished by:

/bin/echo -e "2\n4\ny" | generate_random_files
but it doesn't look particularly readable. Another option would be to create a text file input.txt with contents:
2
4
y
and then invoke the program with redirected input from this file:

generate_random_files < input.txt

This gives you one more file to manage though. Also, it makes it clumsy to use Bash variables in the input. For example, if you want a file size not of 4, but of some variable $calculatedSize, then you would have to manipulate the text file from within your script before calling generate_random_files.

An elegant way to do it is to use what is called "here documents", which allows you to basically put the literal contents of input.txt right in your script, eliminating the need for a separate file and allowing use of variables, while keeping things reasonably human-readable. The syntax is: program name followed by << followed by a unique delimiter of your own choosing, then on a new line the text that serves as input for the program, and then on a new line (and not preceded by any spaces or tabs!) the delimiter again. Like in this little script:


#!/bin/bash
echo "Starting generate_random_files..."
generate_random_files <<TextForInput
2
4
y
TextForInput
echo "Finished!"
And here's the same example, but using a variable which holds the size in bytes, calculated based on the argument $1 that provides the size in kilobytes:

#!/bin/bash
sizeInKb=$1
calculatedSize=$((1024*$sizeInKb))
echo "Starting generate_random_files..."
generate_random_files <<TextForInput
2
$calculatedSize
y
TextForInput
echo "Finished!"