Gaslamp 1.0.4.1 Update

Gaslamp has been updated with a few minor fixes. I’ve switched to a different Geolocation provider so you can continue to acquire location and sunrise/sunset times automatically. Also fixed the “Configure” link in the menu.

In addition to this update, I’ve made some graphical changes. The new logo is here finally here as well as a new web browser icon.

Prizm 1.0 now available!

Prizm is a fully functional .NET library for working with color spaces with numerous features.

This is the first update I’ve posted in quite some time. There are several new articles I have had scheduled but due to their time-consuming nature they were put on the back burner so I could focus on my open source efforts. If you have not visited my Github profile, that is where I tend to focus most of my attention these days. I do however plan to update this site more in the next couple months including a new logo I’ve designed as well as the afforementioned new articles.

Secure Synchronization using SSH

Allows you to synchronize a directory over a secure shell connection. It also has functionality to allow you to run commands remotely, all without leaving the comfort of your local machine.

Installation

    1. Ensure Python 3 and pip are installed.
    2. Install with the following command
    pip install -r requirements.txt
    3. Run with python secure-sync.py

Usage

See secure-sync.py --help for help with parameters.

Running Commands Remotely

You can run arbitrary commands on the remote machine by populating your local directory (created by secure-sync) with a .command file. secure-sync will recognize this file and will cd to the equivalent directory on the remote machine and execute the commands in this file line by line, piping their output to your local machine.

Download

secure-sync.zip 11/04/2017 7 Kb

Dropbox Folder Restoration using Python

Restores any dropbox folder to a previous state. If a file did not exist at the specified time, it will be deleted. Handles unicode parameter input and processing of subfolders.

Example

To restore the folder /photos/tokyo to the way it was on March 7th, 2016:
python dropbox-restore.py /photos/tokyo 2016-03-07

Note: the path /photos/tokyo should be relative to your Dropbox folder; it should not include the path to the Dropbox folder on your hard drive. You will be prompted to confirm access to your Dropbox account through your web browser.

Installation

    1. Ensure Python 3.4 or higher and pip are installed.
    2. Install Dropbox Python API with the following command
    pip install dropbox
    3. Download dropbox-restore.py

Additional Notes

If you wish to modify the source code, you must obtain your own API keys from Dropbox and insert them into the APP_KEY and APP_SECRET fields.

Specifying a time is not officially supported because the time zone is ignored currently. However, it seems like Dropbox always uses UTC, so you can try specifying UTC times at your own risk by specifying the date and time in the format YYYY-MM-DD-HH-MM-SS on the command line. Be warned that Dropbox’s documentation does not guarantee that they will always use UTC, so this can break at any time.

Download

dropbox-restore.py 03/07/2016 5.5 Kb

Recursive Folder Synchronization using VBScript

This is a script written years ago that I still find useful; I had a need to apply updates to many local machines that had no network access from an external hard drive. It synchronizes the contents (files and subfolders) of two folders. Each folder is traversed recursively and any missing subfolders and files are copied both ways. If corresponding folders contain files with matching file names but with different timestamps, the file with the oldest timestamp will be overwritten.

Option Explicit

ForceScriptEngine("cscript")

Dim wshArgs
Set wshArgs = Wscript.Arguments
If WshArgs.Count = 2 Then
  Call SyncFolders(WshArgs.Item(0), WshArgs.Item(1))
  ' Also run once in reverse to catch mismatching subfolder count:
  Call SyncFolders(WshArgs.Item(1), WshArgs.Item(0))
Else
  Wscript.Echo("Wrong number of arguments. Syntax: SyncFolders Folder1 Folder2")
  Wscript.Sleep(3000) ' To allow Function syntax popup message to be seen.
End If

Sub SyncFolders(strFolder1, strFolder2)
  Dim objFileSys
  Dim objFolder1
  Dim objFolder2
  Dim objFile1
  Dim objFile2
  Dim objSubFolder
  Dim arrFolders
  Dim i
  Set objFileSys = CreateObject("Scripting.FileSystemObject")
  arrFolders = Array(strFolder1, strFolder2)
  For i = 0 To 1 ' Make sure that missing folders are created first:
    If objFileSys.FolderExists(arrFolders(i)) = False Then
      wscript.echo("Creating folder " & arrFolders(i))
      objFileSys.CreateFolder(arrFolders(i))
    End If
  Next
  Set objFolder1 = objFileSys.GetFolder(strFolder1)
  Set objFolder2 = objFileSys.GetFolder(strFolder2)
  For i = 0 To 1
    If i = 1 Then ' Reverse direction of file compare in second run
      Set objFolder1 = objFileSys.GetFolder(strFolder2)
      Set objFolder2 = objFileSys.GetFolder(strFolder1)
    End If
    For Each objFile1 in objFolder1.files
      If Not objFileSys.FileExists(objFolder2 & "\" & objFile1.name) Then
        Wscript.Echo("Copying " & objFolder1 & "\" & objFile1.name & _
          " to " & objFolder2 & "\" & objFile1.name)
        objFileSys.CopyFile objFolder1 & "\" & objFile1.name, _
          objFolder2 & "\" & objFile1.name
      Else
        Set objFile2 = objFileSys.GetFile(objFolder2 & "\" & objFile1.name)
        If objFile1.DateLastModified > objFile2.DateLastModified Then
          Wscript.Echo("Overwriting " & objFolder2 & "\" & objFile1.name & _
            " with " & objFolder1 & "\" & objFile1.name)
          objFileSys.CopyFile objFolder1 & "\" & objFile1.name, _
            objFolder2 & "\" & objFile1.name    
        End If
      End If
    Next
  Next
  For Each objSubFolder in objFolder1.subFolders
    Call SyncFolders(strFolder1 & "\" & objSubFolder.name, strFolder2 & _
      "\" & objSubFolder.name)
  Next
  Set objFileSys = Nothing
End Sub

Sub ForceScriptEngine(strScriptEng)
  ' Forces this script to be run under the desired scripting host.
  ' Valid arguments are "wscript" or "cscript".
  ' The command line arguments are passed on to the new call.
  Dim arrArgs
  Dim strArgs
  For Each arrArgs In WScript.Arguments
    strArgs = strArgs & " " & Chr(34) & arrArgs & Chr(34)
  Next
  If Lcase(Right(Wscript.FullName, 12)) = "\wscript.exe" Then
    If Instr(1, Wscript.FullName, strScriptEng, 1) = 0 Then
      CreateObject("Wscript.Shell").Run "cscript.exe //Nologo " & _
        Chr(34) & Wscript.ScriptFullName & Chr(34) & strArgs
      Wscript.Quit
    End If
  Else
    If Instr(1, Wscript.FullName, strScriptEng, 1) = 0 Then
      CreateObject("Wscript.Shell").Run "wscript.exe " & Chr(34) & _
        Wscript.ScriptFullName & Chr(34) & strArgs
      Wscript.Quit
    End If
  End If
End Sub

When only one-way sync is needed (only folder 2 is modified), use this version:

Option Explicit

ForceScriptEngine("cscript")

Dim wshArgs
Set wshArgs = Wscript.Arguments
If WshArgs.Count = 2 Then
  Call SyncFolders(WshArgs.Item(0), WshArgs.Item(1))
Else
  Wscript.Echo("Wrong number of arguments. Syntax: SyncFolders Folder1 Folder2")
  Wscript.Sleep(3000) ' To allow Function syntax popup message to be seen.
End If

Sub SyncFolders(strFolder1, strFolder2)
  ' Note: This version only copies from folder1 to folder2
  Dim objFileSys
  Dim objFolder1
  Dim objFolder2
  Dim objFile1
  Dim objFile2
  Dim objSubFolder
  Dim arrFolders
  Dim i
  arrFolders = Array(strFolder1, strFolder2)
  Set objFileSys = CreateObject("Scripting.FileSystemObject")
  For i = 0 To 1 ' Make sure that missing folders are created first:
    If objFileSys.FolderExists(arrFolders(i)) = False Then
      wscript.echo("Creating folder " & arrFolders(i))
      objFileSys.CreateFolder(arrFolders(i))
    End If
  Next
  Set objFolder1 = objFileSys.GetFolder(strFolder1)
  Set objFolder2 = objFileSys.GetFolder(strFolder2)
  For Each objFile1 in objFolder1.files
    If Not objFileSys.FileExists(objFolder2 & "\" & objFile1.name) Then
      Wscript.Echo("Copying " & objFolder1 & "\" & objFile1.name & _
        " to " & objFolder2 & "\" & objFile1.name)
      objFileSys.CopyFile objFolder1 & "\" & objFile1.name, _
        objFolder2 & "\" & objFile1.name
    Else
      Set objFile2 = objFileSys.GetFile(objFolder2 & "\" & objFile1.name)
      If objFile1.DateLastModified > objFile2.DateLastModified Then
        Wscript.Echo("Overwriting " & objFolder2 & "\" & objFile1.name & _
          " with " & objFolder1 & "\" & objFile1.name)
        objFileSys.CopyFile objFolder1 & "\" & objFile1.name, _
          objFolder2 & "\" & objFile1.name    
      End If
    End If
  Next
  For Each objSubFolder in objFolder1.subFolders
    Call SyncFolders(strFolder1 & "\" & objSubFolder.name, strFolder2 & _
      "\" & objSubFolder.name)
  Next
  Set objFileSys = Nothing
End Sub

Sub ForceScriptEngine(strScriptEng)
  ' Forces this script to be run under the desired scripting host.
  ' Valid arguments are "wscript" or "cscript".
  ' The command line arguments are passed on to the new call.
  Dim arrArgs
  Dim strArgs
  For Each arrArgs In WScript.Arguments
    strArgs = strArgs & " " & Chr(34) & arrArgs & Chr(34)
  Next
  If Lcase(Right(Wscript.FullName, 12)) = "\wscript.exe" Then
    If Instr(1, Wscript.FullName, strScriptEng, 1) = 0 Then
      CreateObject("Wscript.Shell").Run "cscript.exe //Nologo " & _
        Chr(34) & Wscript.ScriptFullName & Chr(34) & strArgs
      Wscript.Quit
    End If
  Else
    If Instr(1, Wscript.FullName, strScriptEng, 1) = 0 Then
      CreateObject("Wscript.Shell").Run "wscript.exe " & Chr(34) & _
        Wscript.ScriptFullName & Chr(34) & strArgs
      Wscript.Quit
    End If
  End If
End Sub

Writing Adobe Swatch Exchange (ase) file using C#

In my last post, I described how to read Adobe Swatch Exchange files using C#. Now I’m going to update that sample progtram to save ase files as well as load them.

An example of a multi-group ASE file created by the sample application

An example of a multi-group ASE file created by the sample application

Writing big endian values

I covered the basics of writing big-endian values in my original post on writing Photoshop aco files, so I’ll not cover that again but only mention the new pieces.

Firstly, we now need to store float values. I mentioned the trick that BitConverter.ToSIngle does where it converts a int to a pointer, and then the pointer to a float. I’m going to do exactly the reverse in order to write the float to a stream – convert the float to a pointer, then convert it to an int, then write the bytes of the int.

public static void WriteBigEndian(this Stream stream, float value)
{
  unsafe
  {
    stream.WriteBigEndian(*(int*)&value);
  }
}

We also need to store unsigned 2-byte integers, so we have another extension for that.

public static void WriteBigEndian(this Stream stream, ushort value)
{
  stream.WriteByte((byte)(value >> 8));
  stream.WriteByte((byte)(value >> 0));
}

Finally, let’s not forget our length prefixed strings!

public static void WriteBigEndian(this Stream stream, string value)
{
  byte[] data;

  data = Encoding.BigEndianUnicode.GetBytes(value);

  stream.WriteBigEndian(value.Length);
  stream.Write(data, 0, data.Length);
}

Saving the file

I covered the format of an ase file in the previous post, so I won’t cover that again either. In summary, you have a version header, a block count, then a number of blocks – of which a block can either be a group (start or end) or a color.

Saving the version header is rudimentry

private void WriteVersionHeader(Stream stream)
{
  stream.Write("ASEF");
  stream.WriteBigEndian((ushort)1);
  stream.WriteBigEndian((ushort)0);
}

After this, we write the number of blocks, then cycle each group and color in our document.

private void WriteBlocks(Stream stream)
{
  int blockCount;

  blockCount = (this.Groups.Count * 2) + this.Colors.Count + this.Groups.Sum(group => group.Colors.Count);

  stream.WriteBigEndian(blockCount);

  // write the global colors first
  // not sure if global colors + groups is a supported combination however
  foreach (ColorEntry color in this.Colors)
  {
    this.WriteBlock(stream, color);
  }

  // now write the groups
  foreach (ColorGroup group in this.Groups)
  {
    this.WriteBlock(stream, group);
  }
}

Writing a block is slightly complicated as you need to know – up front – the final size of all of the data belonging to that block. Originally I wrote the block to a temporary MemoryStream, then copied the length and the data into a real stream but that isn’t a very efficient approach, so now I just calculate the block size.

Writing Groups

If you recall from the previous article, a group is comprised of a least two blocks – one that starts the group (and includes the name), and one that finishes the group. There can also be any number of color blocks in between. Potentially you can have nested groups, but I haven’t coded for this yet – I need to experiment with ase files some more, at which point I’ll update these samples if need be.

private int GetBlockLength(Block block)
{
  int blockLength;

  // name data (2 bytes per character + null terminator, plus 2 bytes to describe that first number )
  blockLength = 2 + (((block.Name ?? string.Empty).Length + 1) * 2);

  if (block.ExtraData != null)
  {
    blockLength += block.ExtraData.Length; // data we can't process but keep anyway
  }

  return blockLength;
}

private void WriteBlock(Stream stream, ColorGroup block)
{
  int blockLength;

  blockLength = this.GetBlockLength(block);

  // write the start group block
  stream.WriteBigEndian((ushort)BlockType.GroupStart);
  stream.WriteBigEndian(blockLength);
  this.WriteNullTerminatedString(stream, block.Name);
  this.WriteExtraData(stream, block.ExtraData);

  // write the colors in the group
  foreach (ColorEntry color in block.Colors)
  {
    this.WriteBlock(stream, color);
  }

  // and write the end group block
  stream.WriteBigEndian((ushort)BlockType.GroupEnd);
  stream.WriteBigEndian(0); // there isn't any data, but we still need to specify that
}

Writing Colors

Writing a color block is fairly painless, at least for RGB colors. As with loading an ase file, I’m completely ignoring the existence of Lab, CMYK and Gray scale colors.

private int GetBlockLength(ColorEntry block)
{
  int blockLength;

  blockLength = this.GetBlockLength((Block)block);

  blockLength += 6; // 4 bytes for the color space and 2 bytes for the color type

  // TODO: Include support for other color spaces

  blockLength += 12; // length of RGB data (3 * 4 bytes)

  return blockLength;
}

private void WriteBlock(Stream stream, ColorEntry block)
{
  int blockLength;

  blockLength = this.GetBlockLength(block);

  stream.WriteBigEndian((ushort)BlockType.Color);
  stream.WriteBigEndian(blockLength);

  this.WriteNullTerminatedString(stream, block.Name);

  stream.Write("RGB");

  stream.WriteBigEndian((float)(block.R / 255.0));
  stream.WriteBigEndian((float)(block.G / 255.0));
  stream.WriteBigEndian((float)(block.B / 255.0));

  stream.WriteBigEndian((ushort)block.Type);

  this.WriteExtraData(stream, block.ExtraData);
}

Caveats

When I originally tested this code, I added a simple compare function which compared the bytes of a source ase file with a version written by the new code. For two of the three samples I was using, this was fine, but for the third the files didn’t match. As this didn’t help me in any way diagnose the issue, I ended up writing a very basic (and inefficient) hex viewer, artfully highlighted using the same colors as the ase format description on sepla.net

Comparing a third party ASE file with the version created by the sample application

Comparing a third party ASE file with the version created by the sample application

This allowed me to easily view the files side by side and be able to break them down into their sections to see what was wrong. The example screenshot above shows an identical comparison.

Another compare of a third party ASE file with the version created by the sample application, showing the color data is the same, but the raw file differs

Another compare of a third party ASE file with the version created by the sample application, showing the color data is the same, but the raw file differs

With that third sample file, it was more complicated. In the first case, the file sizes were different – the hex viewer very clearly showed that the sample file has 3 extra null bytes at the end of the file, which my version doesn’t bother writing. I’m not entirely sure what these bytes are for, but I can’t imagine they are official as it’s an odd number.

The second issue was potentially more problematic. In the screenshot above, you can see all the orange values which are the float point representations of the RGB colors, and the last byte of each of these values does not match. However, the translated RGB values do match, so I guess it is a rounding/precision issue.

When I turn this into something more production ready, I will probably store the original floating point values and write them back, rather than loosing precision by converting them to integers (well, bytes really as the range is 0-255) and back again.

 

Download

AdobeSwatchExchangeLoader-v2.zip 10/20/2015 324 Kb

Reading Adobe Swatch Exchange (ASE) files using C#

Previously I wrote articles on how to read and writes files using the Photoshop Color Swatch file format. In this article, I’m now going to take a long overdue look at Adobe’s  Swatch Exchange file format and show how to read these files using C#.

An example of an ASE file with a single group containing 5 RGB colors

An example of an ASE file with a single group containing 5 RGB colors

Note

Unlike some of Adobe’s other specification, they don’t seem to have published an official specification for the ase format. For the purposes of this article, I’ve had to manually poke around in sample files using a HEX editor to undercover their structure.

And, as with my previous articles, the code doesn’t handle CMYK or Lab color spaces.

 

Structure of a Adobe Swatch Exchange file

ase files support the notion of groups, so you can have multiple groups containing colors. Judging by the files I have tested, you can also have a bunch of colors without a group at all. I’m uncertain if groups can be nested, so I have assumed they cannot be.

With that said, the structure is relatively straight forward, and helpfully includes data that means I can skip the bits that I have no idea at all what they are. The format is comprised of a basic version header, then a number of blocks. Each block includes a type, data length, the block name, and then additional data specific to the block type, and optional custom data specific to that particular block.

Blocks can either be a color, the start of a group, or the end of a group.

Color blocks include the color space, 1-4 floating point values that describe the color (3 for RGB and LAB, 4 for CMYK and 1 for Grayscale), and a type.

Finally, all blocks can carry custom data. I have no idea what this data is, but it doesn’t seem to be essential nor are you required to know what it is for in order to pull out the color information. Fortunately, as you know how large each block is, you can skip the remaining bytes from the block and move onto the next one. As there seems to be little difference between the purposes of aco and asefiles (the obvious one being that the former is just a list of colors while the latter supports grouping). I assume this data is metadata from the application that created the file, but this is just a conjecture.

The following table attempts to describe the layout.

Length Description
4 Signature
2 Major Version
2 Minor Version
4 Number of blocks
variable Block data
Length Description
2 Type
4 Block length
2 Name length
(name length) Name
Colour blocks only
Length Description
4 Colour space
12 (RGB, LAB), 16 (CMYK), 4 (Grayscale) Colour data. Every four bytes represents one floating point value
2 Colour type
All blocks
Length Description
variable (Block length – previously read data) Unknown

As with aco files, all the data in an ase file is stored in big-endian format and therefore needs to be reversed on Windows systems. Unlike the aco files where four values are present for each color even if not required by the appropriate color space, the ase format uses between one and four values, make it slightly more compact than aco.

 

Color Spaces

I mentioned above that each color has a description of what color space it belongs to. There appear to be four supported color spaces. Note that space names are 4 characters long in an ase file, shorter names are therefore padded with spaces.

  • RGB
  • LAB
  • CMYK
  • Gray

In my experiments, RGB was easy enough – just multiply the value read from the file by 255 to get the right value to use with .NET’s Color structure. I don’t know the other 3 types however – I need more samples!

 

Big-endian conversion

I covered the basics of reading shorts, ints, and strings in big-endian in my previous article on aco files so I won’t cover that here.

However, this time around I do need to read floats from the files too. While the BitConverter class has a ToSingle method that will convert a 4-byte array to a float, of course it is for little-endian.

I looked at the reference source for this method and saw it does a neat little trick – it converts the four bytes to an integer, then creates a float from that integer via pointers.

So, I used the same approach – read an int in big-endian, then convert it to a float. The only caveat is that you are using pointers, meaning unsafe code. By default you can’t use the unsafe keyword without enabling a special option in project properties. I use unsafe code quite frequently when working with image data and generally don’t have a problem, if you are unwilling to enable this option then you can always take the four bytes, reverse the, and then call BitConverter.ToSingle with the reversed array.

public static float ReadSingleBigEndian(this Stream stream)
{
  unsafe
  {
    int value;

    value = stream.ReadUInt32BigEndian();

    return *(float*)&value;
  }
}

Another slight difference between aco and ase files is that in ase files, strings are null terminated, and the name length includes that terminator. Of course, when reading the strings back out, we really don’t want that terminator to be included. So I added another helper method to deal with that.

public static string ReadStringBigEndian(this Stream stream)
{
  int length;
  string value;

  // string is null terminated, value saved in file includes the terminator

  length = stream.ReadUInt16BigEndian() - 1;
  value = stream.ReadStringBigEndian(length);
  stream.ReadUInt16BigEndian(); // read and discard the terminator

  return value;
}

 

Storage classes

In my previous examples on reading color data from files, I’ve kept it simple and returned arrays of colors, discarding incidental details such as names. This time, I’ve created a small set of helper classes. to preserve this information and to make it easier to serialize it.

internal abstract class Block
{
  public byte[] ExtraData { get; set; }
  public string Name { get; set; }
}

internal class ColorEntry : Block
{
  public int B { get; set; }
  public int G { get; set; }
  public int R { get; set; }
  public ColorType Type { get; set; }

  public Color ToColor()
  {
    return Color.FromArgb(this.R, this.G, this.B);
  }
}

internal class ColorEntryCollection : Collection<ColorEntry>
{ }

internal class ColorGroup : Block, IEnumerable<ColorEntry>
{
  public ColorGroup()
  {
    this.Colors = new ColorEntryCollection();
  }

  public ColorEntryCollection Colors { get; set; }

  public IEnumerator<ColorEntry> GetEnumerator()
  {
    return this.Colors.GetEnumerator();
  }

  IEnumerator IEnumerable.GetEnumerator()
  {
    return this.GetEnumerator();
  }
}

internal class ColorGroupCollection : Collection<ColorGroup>
{ }

internal class SwatchExchangeData
{
  public SwatchExchangeData()
  {
    this.Groups = new ColorGroupCollection();
    this.Colors = new ColorEntryCollection();
  }

  public ColorEntryCollection Colors { get; set; }
  public ColorGroupCollection Groups { get; set; }
}

That should be all we need, time to load some files!

 

Reading the file

To start with, we create a new ColorEntryCollection that will be used for global colors (i.e. color blocks that don’t appear within a group). To make things simple, I’m also creating a Stack to which I push this global collection. Later on, when I encounter a start group block, I’ll Push a new ColorEntryCollection to this stack, and when I encounter an end group block, I’ll Pop the value at the top of the stack. This way, when I encounter a color block, I can easily add it to the right collection without needing to explicitly keep track of the active group or lack thereof.

public void Load(string fileName)
{
  Stack<ColorEntryCollection> colors;
  ColorGroupCollection groups;
  ColorEntryCollection globalColors;

  groups = new ColorGroupCollection();
  globalColors = new ColorEntryCollection();
  colors = new Stack<ColorEntryCollection>();

  // add the global collection to the bottom of the stack to handle color blocks outside of a group
  colors.Push(globalColors);

  using (Stream stream = File.OpenRead(fileName))
  {
    int blockCount;

    this.ReadAndValidateVersion(stream);

    blockCount = stream.ReadUInt32BigEndian();

    for (int i = 0; i &lt; blockCount; i++)
    {
      this.ReadBlock(stream, groups, colors);
    }
  }

  this.Groups = groups;
  this.Colors = globalColors;
}

After opening a Stream containing our file data, we need to check that the stream contains both ase data, and that the data is a version we can read. This is done by reading 8 bytes from the start of the data. The first four are ASCII characters which should match the string ASEFASEF, the next two are the major version and the final two the minor version.

private void ReadAndValidateVersion(Stream stream)
{
  string signature;
  int majorVersion;
  int minorVersion;

  // get the signature (4 ascii characters)
  signature = stream.ReadAsciiString(4);

  if (signature != "ASEF")
  {
    throw new InvalidDataException("Invalid file format.");
  }

  // read the version
  majorVersion = stream.ReadUInt16BigEndian();
  minorVersion = stream.ReadUInt16BigEndian();

  if (majorVersion != 1 && minorVersion != 0)
  {
    throw new InvalidDataException("Invalid version information.");
  }
}

Assuming the data is valid, we read the number of blocks in the file, and enter a loop to process each block. For each block, first we read the type of the block, and then the length of the block’s data.

How we continue reading from the stream depends on the block type (more on that later), after which we work out how much data is left in the block, read it, and store it as raw bytes on the off-chance the consuming application can do something with it, or for saving back into the file.

This technique assumes that the source stream is seekable. If this is not the case, you’ll need to manually keep track of how many bytes you have read from the block to calculate the remaining custom data left to read.

private void ReadBlock(Stream stream, ColorGroupCollection groups, Stack<ColorEntryCollection> colorStack)
{
  BlockType blockType;
  int blockLength;
  int offset;
  int dataLength;
  Block block;

  blockType = (BlockType)stream.ReadUInt16BigEndian();
  blockLength = stream.ReadUInt32BigEndian();

  // store the current position of the stream, so we can calculate the offset
  // from bytes read to the block length in order to skip the bits we can't use
  offset = (int)stream.Position;

  // process the actual block
  switch (blockType)
  {
    case BlockType.Color:
      block = this.ReadColorBlock(stream, colorStack);
      break;
    case BlockType.GroupStart:
      block = this.ReadGroupBlock(stream, groups, colorStack);
      break;
    case BlockType.GroupEnd:
      block = null;
      colorStack.Pop();
      break;
    default:
      throw new InvalidDataException($"Unsupported block type '{blockType}'.");
  }

  // load in any custom data and attach it to the
  // current block (if available) as raw byte data
  dataLength = blockLength - (int)(stream.Position - offset);

  if (dataLength > 0)
  {
    byte[] extraData;

    extraData = new byte[dataLength];
    stream.Read(extraData, 0, dataLength);

    if (block != null)
    {
      block.ExtraData = extraData;
    }
  }
}

Processing groups

If we have found a “start group” block, then we create a new ColorGroup object and read the group name. We also push the group’s ColorEntryCollection to the stack I mentioned earlier.

private Block ReadGroupBlock(Stream stream, ColorGroupCollection groups, Stack<ColorEntryCollection> colorStack)
{
  ColorGroup block;
  string name;

  // read the name of the group
  name = stream.ReadStringBigEndian();

  // create the group and add it to the results set
  block = new ColorGroup
  {
    Name = name
  };

  groups.Add(block);

  // add the group color collection to the stack, so when subsequent colour blocks
  // are read, they will be added to the correct collection
  colorStack.Push(block.Colors);

  return block;
}

For “end group” blocks, we don’t do any custom processing as I do not think there is any data associated with these. Instead, we just pop the last value from our color stack. (Of course, that means if there is a malformed ase file containing a group end without a group start, this procedure is going to crash sooner or later!)

Processing colors

When we hit a color block, we read the color’s name and the color mode.

Then, depending on the mode, we read between 1 and 4 float values which describe the color. As anything other than RGB processing is beyond the scope of this article, I’m throwing an exception for the LAB, CMYK and Gray color spaces.

For RGB colors, I take each value and multiply it by 255 to get a value suitable for use with .NET’s Color struct.

After reading the color data, there’s one official value left to read, which is the color type. This can either be Global (0), Spot (1) or Normal (2).

Finally, I construct a new ColorEntry obect containing the color information and add it to whatever ColorEntryCollection is on top of the stack.

private Block ReadColorBlock(Stream stream, Stack<ColorEntryCollection> colorStack)
{
  ColorEntry block;
  string colorMode;
  int r;
  int g;
  int b;
  ColorType colorType;
  string name;
  ColorEntryCollection colors;

  // get the name of the color
  // this is stored as a null terminated string
  // with the length of the byte data stored before
  // the string data in a 16bit int
  name = stream.ReadStringBigEndian();

  // get the mode of the color, which is stored
  // as four ASCII characters
  colorMode = stream.ReadAsciiString(4);

  // read the color data
  // how much data we need to read depends on the
  // color mode we previously read
  switch (colorMode)
  {
    case "RGB ":
      // RGB is comprised of three floating point values ranging from 0-1.0
      float value1;
      float value2;
      float value3;
      value1 = stream.ReadSingleBigEndian();
      value2 = stream.ReadSingleBigEndian();
      value3 = stream.ReadSingleBigEndian();
      r = Convert.ToInt32(value1 * 255);
      g = Convert.ToInt32(value2 * 255);
      b = Convert.ToInt32(value3 * 255);
      break;
    case "CMYK":
      // CMYK is comprised of four floating point values
      throw new InvalidDataException($"Unsupported color mode '{colorMode}'.");
    case "LAB ":
      // LAB is comprised of three floating point values
      throw new InvalidDataException($"Unsupported color mode '{colorMode}'.");
    case "Gray":
      // Grayscale is comprised of a single floating point value
      throw new InvalidDataException($"Unsupported color mode '{colorMode}'.");
    default:
      throw new InvalidDataException($"Unsupported color mode '{colorMode}'.");
  }

  // the final "official" piece of data is a color type
  colorType = (ColorType)stream.ReadUInt16BigEndian();

  block = new ColorEntry
  {
    R = r,
    G = g,
    B = b,
    Name = name,
    Type = colorType
  };

  colors = colorStack.Peek();
  colors.Add(block);

  return block;
}

 

Done

 

An example of a group-less ASE file

An example of a group-less ASE file

The ase format is pretty simple to process, although the fact there is still data in these files with an unknown purpose could be a potential issue.

However, I have tested this code on a number of files downloaded from the internet and have been able to pull out all the color information, so I suspect the Swatch Buckler library will be getting ase support fairly soon!

 

Download

AdobeSwatchExchangeLoader-v2.zip 10/16/2015 152 Kb

ClockworkPDF Beta Release

Finally it is here. I apologize for the delay as I’ve been working on a couple other projects concurrently. This is the first public and testable version, don’t except it to be stable and production ready. I’ve tested it much as I can but if you encounter problems with certain PDF versions, please let me know. Any feedback, ideas for new features or suggestions are gladly welcome.

Oil Painting Filter in C#

Converting an image to make it look like an oil painting is not only a nice effect, but is also an easy to implement algorithm. This article demonstrates how to create an oil painting effect from an image. The oil painting filter consists of two main components: color gradients and pixel color intensities. Result images express a lesser degree of detail and tend to appear to have smaller color ranges.

Implementation

The following steps are required for the algorithm:

  1. Iterate each pixel – Every pixel forming part of the source image should be iterated. When iterating a pixel determine the neighboring pixel values based on the specified filter size (or radius).
  2. Calculate color intensityDetermine the Color Intensity of each pixel being iterated and that of the neighboring pixels. The neighboring pixels included should extend to a range determined by the Filter Size specified. The calculated value should be reduced in order to match a value ranging from zero to the number of Intensity Levels specified.
  3. Determine maximum neighborhood color intensity – When calculating the color intensities of a pixel neighborhood determine the maximum intensity value. In addition, record the occurrence of each intensity level and sum each of the Red, Green and Blue pixel color component values equating to the same intensity level.
  4. Assign the result pixel – The value assigned to the corresponding pixel in the resulting image equates to the pixel color sum total, where those pixels expressed the same intensity level. The sum total should be averaged by dividing the color sum total by the intensity level occurrence

When calculating color intensity reduced to fit the number of levels specified the formula can be expressed as follows:

colorintensityalgorithm.png

The variables in the above formula can be described as follows:

  • i – Intensity: the calculated intensity value.
  • R – Red: The value of the pixel’s red color component.
  • G – Green:
  • B – Blue:
  • l – Number of intensity levels: the maximum number of intensity levels specified.

Result Image: Filter 15, Levels 30

Oil Painting - Filter 15, Levels 30

The sample source code defines OilPaintFilter method, an extension method targeting the Bitmap class. This method determines the maximum color intensity from a pixel’s neighbors.

public static Bitmap OilPaintFilter(this Bitmap sourceBitmap, int levels, int filterSize)
        {
            Bitmap image;
            ArgbColor[] originalData;
            ArgbColor[] resultData;
            Size size;

            image = sourceBitmap;
            size = image.Size;

            originalData = image.GetPixelsFrom32BitArgbImage();
            resultData = new ArgbColor[size.Width * size.Height];

            int[] intensityBin = new int[levels];
            int[] blueBin = new int[levels];
            int[] greenBin = new int[levels];
            int[] redBin = new int[levels];

            levels = levels - 1;

            int filterOffset = (filterSize - 1) / 2;
            int byteOffset = 0;
            int calcOffset = 0;
            int currentIntensity = 0;
            int maxIntensity = 0;
            int maxIndex = 0;

            double blue = 0;
            double green = 0;
            double red = 0;

            for (int offsetY = filterOffset; offsetY &lt; size.Height - filterOffset; offsetY++)
            {
                for (int offsetX = filterOffset; offsetX &lt; size.Width - filterOffset; offsetX++)
                {
                    ArgbColor result;

                    blue = green = red = 0;

                    currentIntensity = maxIntensity = maxIndex = 0;

                    intensityBin = new int[levels + 1];
                    blueBin = new int[levels + 1];
                    greenBin = new int[levels + 1];
                    redBin = new int[levels + 1];

                    byteOffset = offsetY * size.Width + offsetX;

                    for (int filterY = -filterOffset; filterY &lt;= filterOffset; filterY++)
                    {
                        for (int filterX = -filterOffset; filterX &lt;= filterOffset; filterX++)
                        {
                            calcOffset = byteOffset + (filterX) + (filterY * size.Width);

                            currentIntensity = (int)Math.Round(((double)(originalData[calcOffset].R +
                                                originalData[calcOffset].G + originalData[calcOffset].B) / 3.0 * (levels)) / 255.0);

                            intensityBin[currentIntensity] += 1;
                            blueBin[currentIntensity] += originalData[calcOffset].B;
                            greenBin[currentIntensity] += originalData[calcOffset].G;
                            redBin[currentIntensity] += originalData[calcOffset].R;

                            if (intensityBin[currentIntensity] &gt; maxIntensity)
                            {
                                maxIntensity = intensityBin[currentIntensity];
                                maxIndex = currentIntensity;
                            }
                        }
                    }

                    blue = blueBin[maxIndex] / maxIntensity;
                    green = greenBin[maxIndex] / maxIntensity;
                    red = redBin[maxIndex] / maxIntensity;

                    result.A = 255;
                    result.R = ClipByte(red);
                    result.G = ClipByte(green);
                    result.B = ClipByte(blue);

                    resultData[byteOffset] = result;
                }
            }

            return resultData.ToBitmap(size); ;
        }

Downloads

OilPaintingFilter.zip 13 Nov 2014 137 KB

iTunes 11 Style Scroller

Spent a couple of hours seeing if I could recreate the album scroller found in the store view of the latest version of iTunes. Uses CSS3 transforms for animation and some jQuery to handle interaction. Definitely needs tidying up, and probably only works in WebKit and latest browsers for now.

Click Me