Arduino Template Express – Compiling Core Libraries

I remember spending countless hours compiling the Linux kernel back in my college days.  That experience helped me develop a taste for command line utilities and the productivity they bring to a developer workflow.  As you expand your horizons experimenting with AVR micro-controllers and Arduino boards, the time will come when you will need to compile the Arduino core libraries for a new or existing board.   The reasons could range from experimenting with a board at different frequencies to using other Atmel micro-controllers. This post covers how to add a new development board to the Arduino Template Express environment and then compile the libraries it supports.  This post is part of the Arduino Template Express (ATE) series. You may also be interested in these other posts:

  1. Arduino Template Express – Installation
  2. Arduino Template Express – Creating a Sketch
  3. Arduino Template Express – Creating a Library
  4. Arduino Template Express – Using ATTiny Micro-Controllers

Prerequisite

In order to follow this tutorial you will need to have installed Arduino Template Express (ATE).  You will also need msbuild installed and available in your application path together with a few add-ons.   To install ATE follow the Arduino Template Express – Installation.  Atmel Studio 6.1 is based on Microsoft Visual Studio. My expectation is msbuild 4.0 is installed with Atmel Studio when it installs the dotnet framework, but I cannot verify that is the case because I had Microsoft Visual Studio already installed when I installed Atmel Studio. To check if msbuild is installed open a command window and type the following command.

reg.exe query "HKLM\SOFTWARE\Microsoft\MSBuild\ToolsVersions\4.0" /v MSBuildToolsPath

If reg.exe comes back with unable to find the specified registry key or value, msbuild is not installed.  You will need to download the full Dotnet Framework 4.0 from Microsoft.  You will also need to install the msbuild community task and the msbuild extension pack.

Registering the New Board

Figure 1 contains a list of the libraries and boards supported by  Arduino Template Express (ATE).  For the purpose of this tutorial, I am going to add support for the Arduino Mega 1280 board.

List of supported libraries

Figure 1 – Supported Libraries

The task of compiling core libraries for a new board requires registration of the board in the boards.xml file and registering a reference to the new board in the project for each library you want compile.  Let’s take care of registering the Mega1280 board first.  Since the Mega2560 and the Mega1280 are very similar boards, I am going to use the entry for the Mega2560 as a model.  Follow these instructions:

  1. Locate the directory where ATE is installed.  Typically USER\Documents\Arduino Template Express.  If you are not sure open a command window and type set | find “ATE_HOME”.  It will display the environment variable ATE_HOME which points to the location where ATE is installed.
  2. Open %ATE_HOME%\boards.xml with your favorite editor.  Copy the <BoardList> entry for the Mega2560.  In the original file it begins on line 51 and ends on line 71.
  3. Paste the entry and make the changes illustrated in listing 1.   I have used the description of the board provided in the Arduino site.
    <BoardList Include="Mega1280" >
        <Processor>ATmega1280</Processor>
        <Location>Mega1280</Location>
        <Variant>mega</Variant>
        <Vid>null</Vid>
        <Pid>null</Pid>
        <SkipFrequencies></SkipFrequencies>
        <SkipLibs></SkipLibs>
        <DefaultProgrammer>Mega1280BootLoader</DefaultProgrammer>
        <EnableForProjects>true</EnableForProjects>
        <EnableForLibCompilation>true</EnableForLibCompilation>
        <DefaultSpeed>16</DefaultSpeed>
        <BitmapName>external.mega1280</BitmapName>
        <DataPort>COM5</DataPort> <!-- Use the port supported in your environment -->
       <SupportedFrequencies>16</SupportedFrequencies>
       <Description>
       The Arduino Mega is a microcontroller board based on the ATmega1280 (datasheet).
       It has 54 digital input/output pins (of which 14 can be used as PWM outputs),
       16 analog inputs, 4 UARTs (hardware serial ports), a 16 MHz crystal oscillator,
       a USB connection, a power jack, an ICSP header, and a reset button.
       It contains everything needed to support the microcontroller; simply connect it
       to a computer with a USB cable or power it with a AC-to-DC adapter or battery
       to get started. The Mega is compatible with most shields designed for the
       Arduino Duemilanove or Diecimila.
       </Description>
     </BoardList>

Note that for the graphic that is displayed with the board when the new project wizard is launched I have specified external.mega1280.  This tells the wizard to look in the %ATE_HOME%\Graphics for a file named external.mega1280.png.  The size of the picture must be 150×150 pixels and the format must be png.  Figure 2, illustrates the picture of the board used.  Save it to %ATE_HOME%\Graphics and make sure it is named external.mega1280.png.

Mega1280 Board Figure 2 – Mega1280 Board

Next step is to take care of the default programmer for the Mega1280.  I declare Mega1280BootLoader to be default programmer.  A development board can be configured to support multiple programmers (bootloader,ISP,custom,etc) and the desired programmer activated from the sketch project.  Search for Mega2560BootLoader.  It should be around line 471.  Copy the content from line 471 to line 480 and paste it..  Listing 2, illustrates how the programmer entry should be changed.

   <Programmer Include="Mega1280BootLoader">
        <ComPort>COM5</ComPort> <!-- Use the port supported in your environment -->
        <Protocol>wiring</Protocol>
        <ComPortSpeed>115200</ComPortSpeed>
        <ProgTransfer>avrdude</ProgTransfer>
        <Verbosity>-v</Verbosity>
        <MiscParams></MiscParams>
        <SupportEEPROMTransfer>false</SupportEEPROMTransfer>
        <Comment></Comment>
    </Programmer>

Change the ComPort entry as needed in your environment.  The Programmer entry in listing 2 uses avrdude and the wiring protocol to program the board.  You could also use an ISP programmer if you are not going to use the bootloader.  When the option to upload a sketch is pressed in Atmel Studio, this is the entry that drives how the sketch is uploaded to the development board.  We are now ready to add the Mega1280 board to the Arduino core libraries to complete the process.

Adding the New Board to Target Libraries

The first library we are going to test is the ArduinoCore library.  Open the project %ATE_HOME%\Source\ArduinoCore\ArduinoCore.atsln.  Next open the file Scripts\TagetBoards.xml.  The TargetBoards.xml contains a list of all the boards supported by the ArduinoCore library.  When compiling all supported boards from the command line, the build script loops through the list of supported boards and makes each supported board the active board, changes variables and references according to each board profile and compiles the code.  Finally the compiled code is copied to the %ATE_HOME%\Source\_Stage\BoardName\Libs\Frequency.  Follow these steps to take care of the changes.

  1. Create an entry for a new supported board and name it Mega1280 (the name must match the name used in the boards.xml).  Use the default programmer (Mega1280BootLoader)  and frequencies configure in boards.xml.
  2. Modify the CurrentBoard entry (first one in the file) to also reference the Mega1280 development board.  This step is very important.  By making the Mega1280 the CurrentBoard, the core library will only be compiled for the Mega1280 when you invoke the MakeIt.bat script with the Compile parameter.  If you skip this step, you will need to call MakeIt.bat with the CompileAll parameter which will take a while since the code will be compiled for all boards and it will get to the Mega1280 based on its position in the list.  See Figure 3 for an illustration of how the beginning of the TargetBoards.xml file should look like after the changes are done.  Do not delete the list of all the other boards supported.  We are done with the changes, let’s go ahead and compile the code.
Target Board Registry

Figure 3 – TargetBoard.xml

Compiling Arduino Core

There is a “feature” of msbuild that prevents the scripts distributed with ATE 1.0.2 Beta 3 from running when there are spaces in the path.  I have a fix that will take care of this in the final ATE 1.0.2 release.  If you have ATE 1.0.2 Beta 3, copy the folder %ATE_HOME%\Source to the root of your C:\ drive, you will end up with a c:\source folder.  If you have a folder name source already in your c:\ drive you can provide any name you want as long as it does not have a space.  Follow these instructions:

  1. Open a command window.  All prerequisites discussed at the beginning of the post need to be met for the rest of the instructions to work.
  2. Go to the directory source\ArduinoCore\Scripts and type makeit compile.  This will compile only the current board, which we previously setup to be the Mega1280.  See Figure 4 for results of compilation.

    ArduinoCore Compilation Result

    Figure 4 – Result of ArduinoCore compilation

  3. After compilation is completed, check the directory source\_stage.  You will find a Mega1280 directory with sub-directories name Headers and Libs.  The Headers contains all headers files for the ArduinoCore and future libraries you compile for this board.  The Libs contains a sub-directory for every supported frequency for the board.  In this case 16 (representing 16mhz) is the only directory available.  Inside the 16 directory you will find the ArduinoCore libraries in release and debug modes.  See Figure 5.

    stage folder

    Figure 5 – Stage folder after compilation

  4. Invoking the script makeit compileall will compile the ArduinoCore libraries for ALL boards supported.  This takes a long while and IS NOT required for this tutorial.
  5. The last step requires copying the Mega1280 folder just created to the %ATE_HOME%\Boards folder.  This is the root folders from where Atmel Studio projects pickup libraries for linking sketches.  For instance, when you develop a sketch for the Mega1280 board, libraries are expected to be found at %ATE_HOME%\Boards\Mega1280\Libs\16 and headers are located at %ATE_HOME%\Boards\Mega1280\Headers.  The frequency at the end of the libs directory changes according to the value of the F_CPU variable.
  6. To compile other libraries repeat the steps documented in the above section Adding New Board to Target Libraries.  In fact you can just copy the <CurrentBoard> and <SupportedBoard> entries created for the ArduinoCore library, see figure 3, and paste it into the targetboards.xml of the library you want to compile.  Then go to the source\LibraryIWantToComile\Scripts and run makeit.bat compile.  Look in the source\_stage\Mega1280\Libs\16 and you will find the debug and release versions of the compiled library.

Configuring a new board takes is a bit involved the first time you do it since you will need to configure tools, paths, etc. After you get the environment setup correctly and understand the mechanics of adding new boards and configuring projects to support it, adding future boards will take you a few minutes.

Arduino Template Express – Using the ATTiny Micro-Controllers

Featured

The Arduino core libraries provide a well defined structure for developing code with the different micro-contollers it supports, but not every project requires the power provided by the ATMega 328 or 2560 chips. Simple projects can take advantage of the ATTiny family of micro-controllers. The goal of this post is to demonstrate how to use Arduino Template Express (ATE) to deploy the Arduino Core libraries to the ATTiny family of micro-controllers. This post will show you how to setup a development board with the ATTiny85 micro-controller and enable it with serial communication capabilities and the ability to use the Arduino Core libraries. In order to follow this tutorial, you will need the following:

  1. Atmel Studio 6.1 or greater
  2. Arduino Template Express (ATE) 1.0.2 or greater
  3. An ISP programmer – I will use AVRISP-MKII and AVR Dragon to demonstrate deploying code with either programmer. If you have an Arduino based ATTiny programmer, that is also fine.
  4. FTDI Breakout Board – The FTDI Basic Breakout from SparkFun or the FTDI Friend + extras from Adafruit will do the job. I will use FTDI Basic Breakout from SparkFun to allow the ATTiny85 to send serial information to a terminal. If you are creative and do not have access to a FTDI Breakout board, you can remove the ATMega328 chip from an Uno board and use the serial communication capabilities already included in the board as a substitute for the FTDI Breakout board.
  5. An ATTiny85 assembled on a breadboard or from a development board such as the one provided by Tinkerlog.

Although, I will use an ATTiny85, the instructions applies to any of the ATTiny micro-controller for which ATE provides a compiled Arduino core library. Figure 1, contains a list of supported micro-controllers.

List of supported libraries

Figure 1 – List of Supported Libraries and micro-controllers

Hardware Configuration

Let’s begin by configuring the development board. Figure 2, contains the schematic diagram of the minimum configuration needed to run an ATTiny85. This configuration uses the internal 8 MHZ clock to drive the micro-controller. In order to transfer data to the FTDI breakout board, pin 2 of the micro-controller is connected to the TX pin of the FTDI breakout board. The 5V supplied by the FTDI breakout board on pin 3 provide power to the micro-controller.  Jumper JP1 is used to enable/disable resetting the micro-controller when a terminal connection is made to the FTDI breakout board.  Jumper JP1 should be open when using the AVR Dragon to debug sketches.  If connected, the capacitor C3 causes side effects while stepping through the code.  By the way, having this jumper on the original Uno board would be a welcome feature.   Note the CTS pin of the FTDI breakout board is connected to ground to prevent the FTDI chip from stopping the transmission.

Figure 2 - ATTINY85 Breadboard Diagram

Figure 2 – ATTINY85 Breadboard Diagram

Figure 3 contains a breadboard implementation of the design. Note the usage of breadboard helpers to speed up the assembly of the circuit.

ATTiny85 Breadboard Implemenation

Figure 3 – ATTiny85 Breadboard Implementation

The FTDI breakout board comes in 5V and 3.3V flavors.  Breakout boards with more advance features have a jumper that controls the voltage output and can be switched between 3.3V and 5V.  If you happen to have a 3.3V FTDI breakout board, you will need to introduce an external power supply to drive the circuit.  Figure 4 contains a circuit diagram that uses an external power supply to provide power to the micro-controller.  Note the 3.3V pin of the FTDI breakout board is not connected to the micro-controller.

Figure 4 - ATTINY85 Breadboard diagram with dedicated power source

Figure 4 – ATTINY85 Breadboard diagram with dedicated power source

The circuits illustrated in figure 2 and 4 are functionally equivalent, the only difference is how the micro-controller is powered. It is important to note that in order for the ISP programmer to work, the micro-controller needs to be connected to an independent power source. Both, the 5V provided by the FTDI breakout board or an external power supply will satisfy this requirement.  Before proceeding connect the ISP programmer to the ISP header (J1) and configure the micro-controller fuses. Table 1 contains a list of the values of the different fuses that need to be programmed.
[table “22” not found /]

If you need information abut AVR fuses check Lady Ada tutorial on her site. Figure 5 illustrates the Atmel Studio screen from where the fuses are programmed. See (Tools/Device Programming) or (Ctrl+Shift+P).

Figure 5 - ATTINY85 Fuses as seen by Atmel Studio

Figure 5 – ATTINY85 Fuses as seen by Atmel Studio

At this point we have a development board assembled in a breadboard. The following step is to register the board in Arduino Template Express so the wizard can pick it up when new projects are created.

How ATE Supports Multiple Boards

ATE uses the concept of a development board as an abstraction of the environment for which a developer creates code. Figure 6, illustrates an entity diagram of a development board.

ATE Entity Diagram

Figure 6 – ATE Entity Diagram

Each development board has a series of properties that identify the board.  A programmer is is a logical construct that includes the port used to program the board and the application responsible for transferring the sketch created by a developer.   ATE comes configured with 17 defined boards, 8 of which can be activated during installation. An activated board shows in the list of available boards when the wizard runs to create a new project. ATE supports an unlimited number of boards that can be manually added or activated after installation. In order to enable the board ArduinoTiny85 follow the following instructions:

  1. Open the file boards.xml with your favorite editor. I would recommend Notepad++ if you do not have a favorite editor already.
  2. Find the board “ArduinoTiny85”, around line 247
  3. Change the property <EnableForProjects> to true, it is false by default. This is the flag that tells the ATE wizard to include the board as an option.
  4. Change the programmer to AVRDRAGONISP-AT.  Since I am using AVR Dragon and will use the ISP header and the atprogram.exe provided by Atmel.  If you are using another programmer, find a suitable entry or create one.  Figure 7 illustrates a modified record.
  5. Configure the DataPort field to reflect the Com Port that is created when you connect the FTDI Breakout Board.  Check the Device Manager list of Com Ports if necessary.
  6. Last, locate the programmer entry AVRDRAGONISP-AT and make sure the ComPort property is set to avrdragon and the protocol to ISP.  ATE 1.0.2 beta was shipped with the wrong port/protocol combination.  This will be changed in the final release.
Editing boards.xml

Figure 7 – Editing boards.xml

At this point we have a development board configured and registered.  Let’s proceed to create a hello world sketch.

Creating Hello World

Start by launching Atmel Studio, selecting new project, and select Arduino Sketch, see figure 8.  Once you press OK the ATE Wizard will start.

atmel-new-project

Figure 8 – New Project Dialog

 Figure 9 illustrate what the wizard will show you.  The boards will vary based on what you have installed, but the ArduinoTiny85 should be in the list.  Select ArduinoTiny85 and also select 8 Mhz from the supported speed drop down.  Press OK and a new project will be created tailored to the Attiny85.

ate-wizard-arduinotiny85

Figure 9 – ATE Wizard with Attiny85 Board

Compile the program and you will get two errors and 4 warnings.  The errors are related to the fact the Arduino Serial library is not supported by the ATTiny85, that’s OK, let’s get rid off all Serial references and concentrate first on making the LED blink.  We’ll introduce a Serial library replacement later.  Comment all lines referencing Serial in main.cpp and Utility\buildinfo.cpp.  Do not delete the lines, just comment them.  At this point you should be able to compile the code and receive 4 warnings, but the code will compile successfully.  Make sure your board has power and the programmer is connected to the ISP header.  Press Deploy in Atmel Studio and see the LED start flashing.  If this the LED does not flash, retrace your steps to verify everything is connected properly.  It is important to note the code is Arduino friendly and you can move to any Ardunio board for which the core libraries are compiled.  This means you are capitalizing on your investment to learn and use the Arduino core library.  Now the LED is flashing and the code can be successfully deployed, let’s take care of the Serial connection.

Enabling Serial Connection

The TinyDebugSerial library provided as open source by Rowdy Dog Software is an implementation compatible with the Serial library provided by Arduino.  It only uses one pin to transmit and does not provide data receive capabilities.  Download TinyDebugSerial.cpp, TinyDebugSerial.h, TinyDebugSerial9600.cpp and core_build_options.h.  Include the “TinyDebugSerial.h” in main.cpp and “..\TinyDebugSerial.h” in BuildInfo.cpp.  Remember to remove the comments on the lines of code referencing Serial.  Recompile and deploy the code.  Open a terminal and connect to the proper COM Port and you will see output coming from the Attiny85.  If you do not want to deal with all of this, a fully functional sketch is included in the link below.  Download, compile and deploy.  If you have configured the ArduinoAttiny85 properly in your board, the deployment script will pick all your local parameters and deploy successfully.

Download Source Code: Sketch85

Arduino Template Express – Creating a Library

In previous posts I discussed How to Install Arduino Template Express and How to Create a Sketch.  In this post let’s discuss how to create a library using Arduino Template Express and what are the advantages and disadvantages of partitioning your code into libraries.  Let’s go!

Divide and Conquer

In order to understand the benefits of libraries, let’s begin by creating the same hello world sketch we created in the How to Create a Sketch post.  Follow those instructions and come back when you have a working sketch in place. Let’s take a look at the main.cpp file to discuss what is going on in the sketch.  See Listing 1.

#ifndef SKETCH_H_
#include "Sketch.h"
#endif
/* Running counter */
static unsigned long counter = 0;
//#pragma message "Sketch Running At: " XSTR(F_CPU) " HZ"
void setup()
{
/* Setup pin connected to board LED for output */
pinMode(LED, OUTPUT);
/* Initialize serial interface */
Serial.begin(SERIAL_SPEED);
/* Show build information via serial interface */
#if defined(ENABLE_TEXT_MESSAGE)
ShowBuildInfo();
#endif
}
void loop()
{
digitalWrite(LED, HIGH);
delay(DELAY_LENGTH);
digitalWrite(LED, LOW);
delay(DELAY_LENGTH);
Serial.print(MESSAGE);
Serial.println( ++counter );
}

Listing 1 – Hello World Sketch

This is a monolithic sketch.  Everything it does is packed into one single development unit.  The sketch uses the Serial library and functions from the core Arduino library, but the functionality of flashing the LED and sending notifications via the UART are packaged in a single program.  As the code stands it is truly a one developer job and contains the following structural (not functional) deficiencies.

  • Unit Testing this code is very difficult.  At best you will be able to verify the LED goes on an off, but testing for LED initial conditions and controlling the timer is very difficult.
  • Having multiple developers work in this program is not possible.  Everything is packed into a single unit of work (main.cpp).
  • Reusing any code of value you develop in this sketch is not possible.  At best you will be able to use clipboard inheritance (copy and paste for the object oriented challenged).
  • Replacing the on board LED by an external LED driven from a II2C/SPI chip requires a great deal of changes because main.cpp is operating too close to the metal by invoking digitatWrite directly.

Let’s visualize how a library can help with our design by introducing a layer of abstraction.  If this was production code, I would make sure there is also a layer of abstraction for the serial communication and for the timer.  For the purpose of this example I will concentrate on the LED.  The sketch main job is manipulating the state (ON|OFF) of a pin by calling digitalWrite().  Let’s introduce the concept of a Led class with two functions on and off.  Figure 1 illustrates the class diagram of the ILed interface and possible implementations.

class-diagram-led

Figure 1

Programming to the ILed interface allows swapping different implementation transparently at compile or run time.  Instead of calling digitalWrite directly the sketch will invoke ILed.on() or ILed.off() to turn the LED on or off.  What are the advantages and disadvantages of introducing this level of abstraction?  I am glad you asked.  Let’s start with the advantages.

  • Programming to an interface promotes code reuse and in this case allows the design of the sketch to evolve to support an Led directly attached to the micro controller or attached to an external component with which the micro controller can communicate.  Besides SPI or II2C you could turn on and off an LED via a XBee,  WiFi, or radio interface.  The possibilities are multiple as long as you implement the ILed interface.
  • Unit Testing an ILed implementation is possible and independent of the main code.  Once an implementation is tested it can be shared and reuse in binary form with a high degree of confidence.  This promotes good design practices such as the Open/Closed principle.
  • Multiple developers can work now on this sketch.  One in the main.cpp sketch and the others in the implementation of the Led, SPILed, II2CLed classes.  The developer responsible for main.cpp will integrate to the different implementation as they become available.  The size of this sketch does not warrant such a division of labor, but in a the context of a bigger project, this option is a plus.
  • The code becomes more cohesive, a desirable trait, this benefits a few “ilities” such as maintainability, reliability, reusability, and understandability.  See Cohesion Metrics for Predicting Maintainability of Service-Oriented Software and Software Design Metrics for Predicting Maintainability of Service-Oriented Software.

Not all is good news, the introduction of ILed in the code has the following disadvantages:

  • After introducing ILed and the Led implementation, the size of the sketch is bigger.  The introduction of abstract classes (the interface of the class diagram) carries a virtual table that takes more memory than the direct digitalWrite approach.  In fact according to avr-size, the primary memory usage of the release version of the sketch goes from 3,176 bytes (9.7%) to 3,676 bytes (11.2%), while the data usage goes from 267 bytes (13%) to 296 bytes (14.5%).  This increment in size is not related to using a library, but to the introduction of ILed into the design.
  • The solution is more complex.  There are more moving parts and more code to develop in order to provide an abstraction layer for the LED.

Let’s refactor the code to introduce a library.  To add a library project to the solution, right-click on the solution and select add/project.  The new project dialog appears.  See Figure 2.  Select Arduino Library Wizard, provide the name Led and select the default location to save it.  Follow the wizard and select the same development board you previously selected for the sketch.  This will create a new library project named Led in your existing solution.

add-new-project-dlg-with-lib-selected

Figure 2 – Adding new library

You can download the full solution with the sketch and the library at the end of the post.  All paths for the sketch to find the library have been added using relative paths.  This is a manual step that I expected Atmel Studio to take care of when I added the library as a reference to the sketch, but unfortunately it did not happen.  See Listing 2 for the new sketch code.

#include

/* Running counter */
static unsigned long counter = 0;

/* Micro-controller connected LED */
Led led;

/***************************************************
Return LED as ILed to isolate caller from what
type of ILed is used
****************************************************/
ILed* getLed()
{
	return &led;
}

 void setup()
 {
	 /* Initialize LED */
	 led.Init(LED);

	 /* Initialize serial interface */
	 Serial.begin(SERIAL_SPEED);

	 /* Show build information via serial interface */
#if defined(ENABLE_TEXT_MESSAGE)
	ShowBuildInfo();
#endif

 }

 void loop()
 {
	 getLed()->On();
	 delay(DELAY_LENGTH);
	 getLed()->Off();
	 delay(DELAY_LENGTH);
	 Serial.print(MESSAGE);
	 Serial.println( ++counter );
 }
 

Listing 2 – Hello World Sketch with ILed

Line 35 and 37 take care of turning the LED on and off.  Reading the code makes it clear you are dealing with an LED.  I could have created a global ILed pointer and initialize it during setup(). The getLed() function provides an opportunity for putting logic that may switch at run time the type of LED the sketch controls.  The source code for the solution with both projects can be downloaded from here.

For Arduino developers making the transition to Atmel Studio, I hope this post has given you an introduction to how to use libraries in your projects. Coming from the Arduino IDE, an environment that promotes code reuse as source code, hides the true advantages of using libraries to decouple functional areas of an application.

Arduino Template Express – Installation

The Arduino ecosystem consist of an IDE, development boards, development libraries and a vibrant and engaging community involved in the support and expansion of all of the components that make the ecosystem.  I acknowledge the pedagogical benefits of the Arduino IDE but if you are already used to Atmel Studio, the Arduino IDE feels like developing with training wheels.  The Arduino Template Express (ATE) enables Atmel Studio to create sketches using Arduino libraries and development boards.  AVR developers can take the best of the Arduino and continue using Atmel Studio and all the benefits it provides.  This post will walk you through the installation of ATE.

Arduino Template Express Configuration

In order to install ATE you must have already installed Atmel Studio 6.1 and Arduino 1.0.4 or higher.  Both of these programs must be installed and running successfully.  Download ATE from the Arduino Template Express site at CodePlex.  Before running setup make sure you close all instances of Atmel Studio that may be running, run atesetup.exe and agree to let the program run with administrative rights.  The following screens will guide you to the setup process.  As of the writing of this post version 1.0.2 (Beta 3) is available for download.

ate-screen1

Press Next to get to the next screen

ate-screen2

Press “I Agree” to accept the license and move to the next screen.

ate-screen3

The atesetup.exe program will install pre-compiled Arduino libraries, project templates and a wizard to drive the initial configuration of sketches.  Click Next to continue.

ate-screen4

This screen let’s you select the directories where Atmel Studio accepts new templates and the location where you want to store the application.  It is advisable to install it in your documents\Arduino Template Express folder.  The installer will try to pre-populate the directories for you.  Click Next to continue

ate-screen5

The following screen will attempt selects the location of AVRDUDE and its configuration file also the path for atprogram.exe distributed with Atmel Studio.  Note that as of Atmel Studio 6.1 the path of atprogram.exe has changes and now it is under the atbackend directory.  The installer will try to find these programs for you, but if does not, you will need to provide their paths before continuing.  Press Next to continue.

ate-screen6

The Board Selection screen allows you to select up to 8 boards you already own.  Do not worry, if you have more than 8 boards.  After the installation is completed you can add as many boards as you want to the boards.xml file.  In my case I have 3 development boards, an Uno, Mega2560 and Leonardo.  Note the Leonardo board requires two different ports, one for data and the other for programming.  Once you have defined all your boards, press Next to continue.

ate-screen7

The External Tool configuration configures the option to deploy sketches from within Atmel Studio. The option is configured in the Tools menu and it is recommended that you configure the Deploy option as part of one of the tool bars in order to enable one click deployment.  Your screen may be a little different depending on the number of external tools you may have already installed in Atmel Studio.  Click Next to continue.

ate-screen8

The About to Install screen gives you a list of what is going to be installed.  Click Install to continue and after a little while the end of the installation screen will show giving you the option to launch the readme file or go to the web site to get additional information.  I suggest you take a quick look at the readme file.

ate-screen9

That’s all.  Hit close and the installation is completed.  If you forget where is the program installed, open a command window and type set | find “ATE_HOME” to find the path where the ATE was installed.  Next post I will walk you through the generation of a sketch that blinks the on board LED and uses the serial port.

Arduino Programming Using Atmel Studio 6.0

This article is no longer relevant.  Please follow the Arduino Template Express update to find a more mature offering to allow development of Arduino sketches in Atmel Studio 6.1.

The Arduino IDE is a good development environment to get started with microcontrollers.  It holds your hand and hides complexities that get in a novice way.  As you gain experience and want to use advance features such as refactoring, assembly language or binary libraries, working with the Arduino IDE just doesn’t cut it.  Moving to a more advance IDE is the next logical step for a mature Arduino user.  It is the equivalent of leaving your home to go to college. Jaycon Systems has a well written three part tutorial explaining how to configure Atmel Studio 6.0 to compile Arduino projects. I have taken a different approach to solve this problem and have created an installation script to accomplish the following;

  1. Distribute a binary file for the core library (libcore).
  2. Distribute binaries of the Arduino libraries (libEEprom, libEthernet, libFirmata, libLquidCrystal, libSD, libServo, libSoftwareSerial, libSPI, libStepper, libWire).
  3. Distribute a solution with projects for all the libraries specified above so libraries can be recompile and redeployed as needed.
  4. Distribute a project template for Arduino Sketch.  This allows you to do a new project and select Arduino Sketch from the list of available templates.
  5. Distribute a project template for Arduino Lib.  This allows you to do a new project and select Arduino Library from the list of available templates.
  6. Distribute a deployment scripts to write sketches to the destination chip.  This allows a one click deployment to the Arduino board.

The rest of this post covers the configuration of the installation script and the deployment scripts in the Atmel Studio environment.

Configuring Installation Script

In order to run the installation script, you need Arduino 1.0.1 or greater and Atmel Studio 6 or greater.  Both of these programs must be installed and running successfully.  The installation script is located at the Arduino Installer site at Codeplex.  It contains all the binaries and scripts required for the installation.  Follow these instructions carefully for a successful installation.

  1. Download the installation package from Codeplex
  2. Unzip in a directory of your choice.  For the purpose of this post I am going to assume you have decompressed the installation script at c:\arduino-install.
  3. Open the file c:\arduino-install\install.bat with your favorite text editor and go to the :SetUserVariables label, around line number 31
  4. By default the ATMEL_CPU is configured to ATmega328p, change this line if necessary to reflect your microcontroller.  Typical values are ATmega32u4 (Leonardo),  ATMega328p (Uno), ATmega2560 (Mega), ATmega168 (Diecimila).
  5. The CPU_FREQUENCY and ARDUINO_VERSION do not need to be changed.  I have tested with the  Arduino 1.0.1 IDE successfully with these default values CPU_FREQUENCY=16000000 and ARDUINO_VERSION=100
  6. Set the COM_PORT variable to the communication port you are using to communicate with the Arduino board.  The default is COM3.
  7. Set the COM_PORT_SPEED variable to the proper value if necessary.  The default is 115200.
  8. The VARIANT variable refers to the pin layout of the board.  Change it accordingly to reflect the type of board you have.  The values are standard (Uno, Diecimila), leonardo (Leonardo), etc.  The default value is standard.
  9. Last setup the AVRDUDE_ROOT_DIR variable with the value of the root director where the avrdude is located.  This could be done from the Arduino installation or from Winavr.  Note that the avrdude.exe would be at AVRDUDE_ROOT_DIR\bin\avrdude.exe from the path you specify.
  10. Save the file and open a command window.  Go to the directory c:\arduino-install and run install.bat.

After a few seconds you should see an output similar to that illustrated on Figure 1.  The installation script has done the following;

Installation Results

Figure 1 – Installation Results

  1. Create directory structures at C:\users\YOUR_USER_NAME\Documents\Atmel Studio\Libs and Headers.
  2. Under Libs there are three directories, ArduinoCore for the core library, Arduino for all the Arduino libraries and Public for your personal libraries.
  3. Under Headers there are three directories, ArduinoCore for the core library headers, Arduino for the different Arduino library headers and Public for your personal library headers.
  4. All binaries and header files have been deployed to these directories.
  5. There is a file named deploy.bat located at c:\users\YOUR_USER_NAME\Documents\Atmel Studio.  This file is responsible for performing generic solution and project name clean up and then invoking the localdeploy.bat file that is located in the Arduino project directory and is part of the sketch template.
  6. In the c:\users\YOUR_USER_NAME\Documents\Atmel Studio\Templates\ProjectTemplates there are two files.  Arduino Library.zip and Arduino Sketch.zip, these are the library and sketch templates for Atmel Studio.
  7. At c:\users\YOUR_USER_NAME\Documents\Atmel Studio\ArduinoLibrariesSource a solution named ArduinoLibraries.atsln has been deployed with the source code for all the Arduino libraries so you can recompile them from within Atmel Studio if you wish to update the distributed binary libraries.

Configuring Deployment Script

The configuration of the deployment script allows you to do deployment of Arduino sketches with one click or keyboard shortcut.  Sketches are deployed to the Arduino board while libraries are deployed to the Public folder of the library directory created by the installation script.  Follow these instructions to configure the deployment script in Atmel Studio;

  1. Open the External Tool manager from Tools/External Tools
  2. Setup a new entry named &Deploy with the characteristics illustrates in Figure 2.

    Deploy Tool Definition

    Figure 2 – External Tool Definition

  3. The & before the D, makes it a hot key while in the context of the Tools menu.  This means pressing ALT-T followed by D will launch the Deploy script.
  4. The parameters must be $(SolutionDir) $(ProjectDir) $(ProjectFileName).  There is a space between each parameter.
  5. Select use Output window so you can see the outcome of the deployment within Atmel Studio.
  6. The top most entry is External Tool 1, the second one is External Tool 2 and so on.  Moving the entries up or down changes the External Tool to the command located at the target position.  If you have multiple external commands registered, keep track of the position where Deploy is located because we are going to need it later.

Let’s create a custom toolbar and add some commands that will facilitate the development experience.

  1. Go to view\Toolbar\Customize and create a new toolbar.
  2. Name the toolbar Productivity.  Hit OK and make sure the checkbox is selected.

    Productivity Toolbar

    Figure 3 – Creating a Productivity toolbar

  3. Select the Commands tab and select the Toolbar radio button.
  4. In the drop down select the Productivity toolbar we just created.
  5. Add the command Build\Build Solution
  6. Add the command Tools\External Command1 (or whatever the name of the Deploy command is in your setup)
  7. Move it up and down as needed.

    Toolbar Commands

    Figure 4 – Toolbar Commands

  8. Click on the Keyboard button and select the Tools.ExternalCommand1
  9. Enter a shortcut for it and press Assign.  In my case I configured the Tools.ExternalCommand1, which is the Deploy script as CTRL+SHIFT+P

    External Tools Shortcut

    Figure 5 – External Tools Shortcut

  10. Hit OK and we are done.  You should see a toolbar with the following information.

    Deploy Toolbar

    Figure 6 – Productivity Toolbar

The option to deploy is available via a click on the Deploy button in the Productivity toolbar or via CTRL+T followed by D or via CTRL+SHIFT+P.  Any one of these three options should satisfy your workflow.

Creating Arduino Sketches

To create your first Arduino sketch in Atmel Studio 6, launch Atmel Studio, select File\New\Project.  The new project window appears similar to figure 7 with all the installed templates.

Atmel Studio New Project

Figure 7- Atmel Studio With Arduino Templates

Select Arduino Sketch, change the name to TestSketch1, change the location if necessary and press OK.  A new solution is created with a project named TestSketch1.  The project has three files, with the following content;

  1. main.cpp – contains code to blink the led on the Arduino board.  This is so you can have a quick block of functional code.
  2. localdeploy.bat – this is the local deployment script.  This file is called by the deploy.bat script located in your c:\users\your_user_name\Documents\Atmel Studio\deploy.bat.  It gives you the opportunity to customize deployment for a particular sketch.  The localdeploy.bat is ready to deploy the hex file located in the Release directory, not debug.  This script reflects the port, speed and directory information provided to the installation script.  I have submitted internal enhancement request AVRSV-3699 to Atmel to have the $(Configuration) variable exported to indicate you are running in Release or Debug mode.  Until this issue is addressed the default hex file deployed is the one in the release directory. You can change the script to deploy from the Debug directory if necessary.
  3. readme.txt – This file contains a list of the Arduino libraries, which files you need to include and which libraries you need to add to the linker.

Before compiling the code you need to open the project properties (ALT-F7),  select All Configurations and under Toolchain\AVR/GNU Linker\Libraries add libcore.  See Figure 8.  There is a bug in Atmel Studio 6 that does not include the libraries added to a project template.  I have submitted internal bug AVRSV-3701 to Atmel for this issue.  Note the Library search path is automatically populated with the directory structure where the libraries are located. After adding the library you can go ahead and compile the code, then press the deploy button in the Productivity Toolbar.  You will see avrdude copy the hex file to the Arduino board and the onboard LED start flashing.

Add Core Library

Figure 8 – Add Core Library

Creating Arduino Libraries

To create an Arduino library, select File\New\Project and select the Arduino Library template from the list of available templates.  Name the library TestLib1 and change the directory as necessary.   A new directory is created with four files.

  1. foo.h and foo.cpp contain a dummy class
  2. localdeploy.bat is the deployment script responsible for deploying the library and header files to the Public directory of the Lib and Header created during installation.  As before only the Release version is deployed for the same reasons previously stated.
  3. readme.txt contains a blank file.

If you need a reference to libcore in your library you must add it as explained for the sketch.  Deploying the library copies all the header files and *.a files to the Public directories.

Conclusion

Adopting Atmel Studio has the potential to increase your productivity and expose you to advance concepts such as mixed language programming (C/C++ and Assembly).  If you are a Microsoft developer familiar with Visual Studio, you will feel right at home in Atmel Studio 6.  Let me know if the script has been of assistance transitioning out of the Arduino IDE.

Capturing Your Project Vision

A hobby is by definition an activity done regularly in one’s leisure time for pleasure.  I am an electronic hobbyist and most of the electronic projects I work on come to fruition as the result of a learning experience.  Rarely, I get to identify a clear need and start working on a solution for it.  The incubation process is more like this; I find an interesting subject as a result of a magazine article, blog post or new product.  I work on a prove of concept to understand the subject and during this stage ideas come to mind where to apply the new concept.  This leads to the creation of a project and on weekends, I get to work on it until completion.  The approach to the creation of a personal project is significantly different from a commercial environment in which an idea must provide a financial or strategical advantage in order to become a project.  Independently of how you get to conceive new projects, they all require a vision.  The goal of the vision is to capture at a high level what is going to be done and who is the beneficiary.  The vision helps you, a team of one, to stay focus on the goals of the project and prevent you from going into the wilderness following new topics of interest that may come up while working on a project.  The purpose of this post is to discuss five different styles to capture a project’s vision. I have picked one project from my backlog to drive the discussion.  The project is a breadboard tool to assist electronic hobbyists in prototyping new projects.

Problem Statement Style

This style presents the vision of a project as a problem to be solved.  It identifies the affected parties and what is the solution the project provides to the problem.  Here is the vision expressed as a problem statement.

[wpc_red_box]The problem of offloading away from the primary micro-controller the functionality to drive LCD displays, manage keypad input, emulate single/array of I2C devices, drive arrays of LEDs and manage arrays of seven segment displays while prototyping a new project on a breadboard affects advance and novice electronic hobbyists.  The impact of which is longer prototyping time, additional complexity, duplication of effort, abandoned projects and wasted time building one off components to support prototyping efforts.  The Breadboard Butler would address these problems by providing the following features; It would be released under a non viral open source/hardware license. It would be reprogrammable to support different type of services. It would provide services for driving LCD displays, keypad input, emulating I2C devices, driving LEDs and driving seven segment displays.  It would create an online repository where services could be shared between users.  It would run from a breadboard. It would use AVR technology. It would allow the primary micro-controller to access services using the I2C protocol to minimize the number of pins required for integration. It’s implementation cost would be less than $US 20.00.[/wpc_red_box]

Free Form Style

This style does not follow a particular template.  It allows you to tailor the vision to the project at hand.  Here is the vision expressed in free form style.
[wpc_red_box]The goal of this project is to increase the productivity of electronic hobbysts when prototyping new projects on a breadboard by proving a solution with the following features; It would be released under a non viral open source/hardware license. It would be reprogrammable to support different type of services. It would provide services for driving LCD displays, keypad input, emulating I2C devices, driving LEDs and driving seven segment displays.  It would create an online repository where services could be shared between users.  It would run from a breadboard. It would use AVR technology. It would allow the primary micro-controller to access services using the I2C protocol to minimize the number of pins required for integration. It’s implementation cost would be less than $US 20.00.[/wpc_red_box]

Business Opportunity Style

This style has some similarities with the problems statement style.  Typically when expressing the vision as a business opportunity a contrast is drawn between the product to be developed and other products in the market.  It is important to be objective and professional when comparing your project with existing ones, specially if they are open source.  Trashing someone’s project to highlight features of your own is not a good practice and in my opinion takes away from the value of your vision.  Here is the vision expressed as a business opportunity;
[wpc_red_box]For advance and novice electronic hobbyists who use any of the following techniques while prototyping a project on a breadboard; LCD displays, Alphanumeric keypads, I2C device simulation, array of LEDs and seven segment displays.  The Breadboard Butler is an open source programmable breadboard add-on that uses AVR technology to provide services to drive LEDs, control LCD displays, keypads or simulate I2C components to improve prototyping efficiency.  Unlike LCD backpack/driver boards using shift registers or GPIOs, the Breadboard Butler uses the I2C protocol to offloads all the processing complexity of providing a service to an independent micro-controller.  This allows the electronic hobbyist to concentrate on the scope of the prototype instead of wasting time building supporting infrastructure.[/wpc_red_box]

Product Box Style

This style allows you to present your vision as a finalized product.  The vision is documented on a product box to contribute to the illusion of a completed product.  The style of box you select is up to your imagination and the type of effect you wish to achieve.    Here is a Photoshop illustration of the vision as a product box.

Mind Map

It has been said that a picture is worth a thousand words.  This style allows you to present your vision in one single picture and categorize features as necessary.  How simple or complex you create your mind map is up to you and how receptive you are to graphical representations.  The mind map below was created with the open source application FreeMind.

Conclusion

I first learned about the problem statement and business opportunity styles around 1997 while using Requisite Pro a product part of the Rational Rose suite, this was before IBM bought Rational Rose and butchered it beyond recognition. Free Form and Product Box styles were picked up as part of different agile methodologies. The Mind Map style was recommended by a member of the AVR Freaks community.  Which style is best to document a project vision? There is not a correct answer to this question.  It depends on which format provides the most information to you or the audience that will read the vision.  The benefits of articulating a clear vision for your project go beyond helping you stay in scope.  If you release your project as open source, a well documented vision will also help future contributors understand short and long term goals of your project.

Downloads

Reversing Bits In A Byte

I am working on a small project that requires interfacing a HD44780 20×4 LCD display to an ATTINY 2313 chip. While designing the PCB for this project, I needed to route the lower nibble of PORTB (PB0-PB3) to the high nibble of the LCD data (D4-D7). Keeping the port mapping straight (PB0->;D4, PB1->;D5, etc) required the addition of at least 6 vias. Reversing the order of the bits ( PB0->;d7, PB1->;D6,etc) required no additional vias, but the content of the lower nibble in PORTB would have to be reverse in code in order to align the data properly. This post documents the analysis I did on different bit reversing routines to determine which one was the most efficient for the task.

I found two viable C implementations to reverse the bits in a byte. Listing 1, illustrates the C code for the reverseByteWithForLoop implementation and the assembly code generated by the compiler. This functions loops through every bit in a byte and reverses every bit that is set. The function is 28 bytes long and takes 92 cycles to run when the data is 0x00 and 100 cycles when the data is 0xFF. It represents 1.37% of the 2K of flash memory in the ATTINY2313.

uint8_t reverseByteWithForLoop( uint8_t num )
{
uint8_t bit;
uint8_t output;
for( int count=1;count>;1;
output = output<;if(bit==1)
output = output+1;
}
return output;
}
/* ===============ASSEMBLY CODE ==================
 ldi	r18, 0x08			; 1 cycle
 ldi	r19, 0x00			; 1 cycle

 mov	r20, r24			; 1 cycle
 andi	r20, 0x01			; 1 cycle
 lsr	r24					; 1 cycle
 add	r25, r25			; 1 cycle
 cpi	r20, 0x01			; 1 cycle
 brne	.+2      			; 1 or 2 cycles
 subi	r25, 0xFF			; 1 cycle
 subi	r18, 0x01			; 1 cycle
 sbci	r19, 0x00			; 1 cycle
 brne	.-20     			; 2 cycles

 mov	r24, r25			; 1 cycle
 ret					; 4 cycle
*/

Listing 2, contains the source code and the assembly code generated by the compiler for the function reverseByteWithShifts. This function is 32 bytes long and takes 19 cycles to run independently of the data. The size of the function represents 1.56% of the available flash memory in the ATTINY2313.

uint8_t reverseByteWithShifts( uint8_t x )
{
x = ((x >;>; 1) &amp; 0x55) | ((x <;<; 1) &amp; 0xaa);
x = ((x >;>; 2) &amp; 0x33) | ((x <;<; 2) &amp; 0xcc);
x = ((x >;>; 4) &amp; 0x0f) | ((x <;<; 4) &amp; 0xf0);
return x;
}
/* ===============ASSEMBLY CODE ==================
 mov    r25, r24            ; 1 cycle
add    r25, r25            ; 1 cycle
andi   r25, 0xAA           ; 1 cycle
lsr    r24                 ; 1 cycle
andi   r24, 0x55           ; 1 cycle
or     r25, r24            ; 1 cycle
mov    r24, r25            ; 1 cycle
add    r24, r24            ; 1 cycle
add    r24, r24            ; 1 cycle
andi   r24, 0xCC           ; 1 cycle
lsr    r25                 ; 1 cycle
lsr    r25                 ; 1 cycle
andi   r25, 0x33           ; 1 cycle
or     r24, r25            ; 1 cycle
swap   r24                 ; 1 cycle
ret                        ; 4 cycle
*/

Listing 3, contains the last function which is my own implementation in assembly language. It is 12 bytes long and uses a loop to reverse all the bits in a byte. It takes this function 37 cycles to complete the task. The size of the function represents 0.59% of the available flash memory in the ATTINY2313.

#if (__GNUC__ * 100 + __GNUC_MINOR__) <; 303
#error &quot;This library requires AVR-GCC 3.3 or later, update to newer AVR-GCC compiler !&quot;
#endif

#include <;avr/io.h>;
.global reverseByte
.func   reverseByte
reverseByte:
ldi        r25, 0x80
rotate_bit:
rol        r24
ror        r25
brcc       rotate_bit
mov        r24, r25
ret
.endfunc

Table 1

Table 1, illustrates the characteristics previously outlined for each one of the functions. This analysis begs the question, which one of these functions is better? Well, it depends. If execution time is the primary concern, reverseByteWithShifts is the best choice at the expense of 1.56% of flash memory. On the other hand, if code size is the driving concern, reverseByte is the best candidate, but it takes twice as long as to run. ReverseByteWithForLoop is smaller than reverseByteWithShifts, but takes the longest to run. Its execution time is dependent on the input data, which makes it a bit more complicated to use in an project that requires accurate timing. To see other alternatives and how this topic evolved over time, visit AVRFreaks. As far as my project goes, reverseByte is the way to go since space concern is more important than execution timing.

Enhancing How Applications Capture Customer Address

Introduction

An accurate customer database is one of the most valuable assets a business can have. Incorrect customer addresses increase the cost of doing business and reduces profit margins. On May 17 of 2009, The United States Post Office published in the National Postal Forum Summit the annual cost of Undeliverable As Addressed (UAA) mail is estimated to be $1.856 billion. 12.4% of this amount is related to addresses with bad elements. Figure 1 contains a breakdown of the different reasons why mail is not delivered. The cost of fixing a customer address varies depending on the complexity and cost of a business process or the stage of the process at which the address discrepancy is found. This article analyses a couple of cost effective and easy to implement solutions to reduce the number of records with incorrect address due to bad address elements.

usps chart

Figure 1, Mail Delivery Failure

Quantify the Problem

For a company like Amazon, a shipment with a bad address represents additional cost and an exception in the business process. For a smaller company like the guys mowing my lawn, a bad address represents a delay in revenue since customers cannot be billed on time. Independently of the business line or application for which you want to introduce the techniques outlined in this article, it is important to understand the scope of the problem and quantify the cost to the business. This information will allow you to measure the effectiveness of the solution at different time periods after implementation. Financial matters aside, let’s discuss specifics.

Typical Address Capture

Figure 2 illustrates a HTML form capturing a customer address. Similar forms can be found in e-commerce web sites or in-house applications. The layout of the form is human friendly and does not give the application an opportunity to proactively fill in address information based on related data elements. The user is required to enter the entire address and at that point the address is ready for validation.

Classic Form

Figure 2 – Classic Address Form

Enhanced Address Capture – Round 1

Figure 3 rearranges the form to allow the application to obtain related data elements as the operator enters information. This approach enhances the user experience and increases the accuracy of the address information. An additional visual element has been introduced to display the address in USPS format. This implementation uses the following data elements relationships.

  • A successful reverse telephone lookup can generate the customer address.
  • A zip code lookup can generate the city and state of the address. This will require the customer/operator to enter only the street address.
Enhanced Form

Figure 3 – Enhanced Form

Enhanced Address Capture – Round 2

Figure 4 provides an implementation aimed to Customer Service Representatives (CSR). It provides a similar address resolution scheme to the previous solution. It also adds value by including an embedded Google map and enhanced spelling for targeted fields using NATO phonetic alphabet. This last feature is activated by placing the mouse over the field label and is a result of observing CSR work with different customers while creating new accounts or making modifications to existing ones. A CSR usually spells back to the customer email, city, address and name. In my experience, the number of fields that get spelled back to the customer depends on factors such as familiarity of the customer or CSR with the English language, speaker accent, quality of phone connection or complexity of the data. For instance, 30 Maine Street can be easily understood as 30 Main Street due to any of the conditions previously listed.

Another advantage of introducing a common spelling solution is the NATO phonetic alphabet can be replaced or modified to include words that mask the CSR accent. This facilitates the communication between customer and CSR and provides consistency to the task of spelling information back to the customer. 30 Main Street will be unambiguously spelled by all CSRs as three, zero, Mike, Alfa, India, November, Space, Sierra, Tango, Romeo, Echo, Echo, Tango.

Full Form

Figure 5 – Full Form

Using the Code

The code is broken in three main tiers. The presentation tier running on the browser and composed of address.html, tutorial.cs, tutorial.js and additional JQuery plugins for general UI support. The middle tier on the web server where the proxy for the reverse telephone lookup runs in the form of an Ajax WCF web service and the data layer where the data provider (whitepages, Google, etc.) stores telephone registration information. The HTML form is setup to run as a singleton in the tutorial.js file.

var theForm = new function MainForm() {
    var instance = null;
    var controller;
    this.getInstance = function() {
        if (instance == null) {
            instance = this;
            init();
        }
       return instance;
    }
    function init() {
        controller = new FormController
		(new InputAddressView(), new FriendlyAddressView());
        controller.init();
    }
}

$(document).ready(function() {
    theForm.getInstance();
});

The initialization of the form takes place during the document ready event generated by JQuery. Two composite views are created, one responsible for capturing user input and managing the map control, the other responsible for displaying the address in USPS format. The controller is responsible for sending/receiving data to/from the middle tier and handling events generated by the InputAddressView control. During initialization, the FriendlyAddressView registers with the controller to receive notification when address information changes. The InputAddressView does not need to register for notification since it is the only view capable of modifying the customer address. The controller initializes the views with default data, configures the reverse phone lookup proxy and notifies all subscribed views the main view has changed. All this displays a form with all fields blank ready for user input.

this.init = function() {
	_inputAddressView.init(THIS);
	_friendlyView.init(THIS);

	addressData = Data.Repository.getNewCustomer()
	setAddressView();
	proxy.setHandler(onTelephoneLookupHandler);
	fire(this);
}

The initialization of the InputAddress dynamically creates each input control (telephone, cellphone, zipcode, etc.) by providing the HTML placeholders where the labels and fields need to be generated. Each control is responsible for generating the input field it represents with the necessary label, masking, length, events, etc.

var _controller;
var zipCode = new CustomTextBox("zipCodeContainer",
	"zipCode3", "", "", "Zip", "", "99999", 5, onZipCodeKeyUp);
var state = new stateComboBox("stateContainer", "state3",
	"StateClass", "", "State", "ZipClass", onStateChange);
var cmbState1 = new stateComboBox("stateContainer1", "state1",
	"StateClass ReadOnly", "DC", "State", "ZipClass");
var cmbState2 = new stateComboBox("stateContainer2", "state2",
	"StateClass ReadOnly", "DC", "State", "ZipClass");
var street = new PhoneticTextBox("streetContainer", "streetAddress3",
	"TextBox", "", "Street Address", "", onStreetKeyUp);
var city = new PhoneticTextBox("cityContainer", "city3",
	"TextBox", "", "City", "", onCityKeyUp);
var fullName = new PhoneticTextBox("fullnameContainer",
	"fullName3", "TextBox", "", "Full Name", "", onFullNameKeyUp);
var email = new PhoneticTextBox("emailContainer", "email3",
	"TextBox", "", "Email", "", onEmailKeyUp);
var cellphone = new telephoneTextBox("cellphoneContainer",
	"cellphone3", "", "", "Cell", "", onCellphoneKeyUp);
var telephone = new telephoneTextBox("telephoneContainer",
	"telephone3", "", "", "Telephone", "", onTelephoneKeyUp);

The telephoneTextBox control is configured to notify the form when the phone field is complete. The form notifies the controller which in turn captures the new telephone number, notifies any view subscribed for notification and requests the reverse phone lookup proxy to perform a lookup.

this.onTelephoneChange = function(sender, data) {
	addressData.setTelephone(data);
	fire(sender);
	proxy.telephoneLookup(data);
}

A successful phone lookup is handled by the onTelephoneLookupHandler method of the controller which captures the new data, updates all views and schedules the map refresh to take place the next time a field loses focus. The map cannot be updated while the user is typing because the iframe where the map is located captures the focus during the map update. Not scheduling the map refresh has the side effect of the current field losing the focus and the data if the field has a mask. The reverse phone lookup takes place while the user is entering the cell phone or email for the customer. Note that the use case dealing with multiple persons registered to a single telephone number is not addressed in this example. The first person registered is used for the example provided.

function onTelephoneLookupHandler(data) {
	if (!data.d.found)
		return;
	addressData.setZipCode(data.d.zipCode);
	addressData.setState(data.d.state);
	addressData.setCity(data.d.city);
	addressData.setStreet(data.d.street);
	addressData.setFullName(data.d.name);
	_inputAddressView.setAddress(addressData, true);
	refreshMapOnChange = true;
	fire(this);
}

Figure 5 illustrates a sequence diagram of the reverse phone lookup. As a side note, the diagram was generated with the on-line sequence diagram generator provided by Web Sequence Diagram.

Sequence Diagram

Figure 5 – Sequence Diagram

The zip code resolution uses a similar approach, except instead of calling a web service hosted in the middle tier, one is called hosted at www.geonames.org. The call is triggered by the controller when the zip code has five digits. A successful lookup captures the city and state associated with the zip code and updates the views accordingly.

this.onZipCodeChange = function(sender, data) {
        var zipText = data.replace("_", "");
        addressData.setZipCode(data);
        if (zipText.length == 5)
            $.getJSON("http://www.geonames.org/postalCodeLookupJSON?&country=
		US&callback=?",
            	{ postalcode: zipText }, onLookupResponse);
        else {
            fire(sender);
        }
    }

function onLookupResponse(response) {
	if (response && response.postalcodes.length &&
		response.postalcodes[0].placeName) {
		addressData.setCity(response.postalcodes[0].placeName);
		addressData.setState(response.postalcodes[0].adminCode1);
		fire(this);
		setAddressView();
		_inputAddressView.focusOnStreet();
	}
}

The reverse telephone lookup is handled by the getAddress method of the web service. Multiple components implementing the IAddressRepository interface are provided to experiment performing reverse phone lookups against WhitePages, Google and AnyWho. Each connector submits the data to the site and parses the returned HTML file to get the address information. Note, the usage of public websites to screen scrape telephone information is not recommended for critical applications and may violate the terms of use of the targeted site. Consider the usage of this technique in the context of the provided code as a pedagogical instrument.

[OperationContract]
public Address getAddress(string telephone)
{
	PhoneRepository.IAddressRepository repository =
				new WhitePagesAddressRepository();
	IList<PhoneRepository.Address> list = new List<PhoneRepository.Address>();

	try
	{
		list = repository.getAddress(telephone);
	}
	catch( System.Exception ex )
	{
	}
	Address newAddress = new Address();
	if (list.Count > 0)
	{
		newAddress.city = list[0].city;
		newAddress.state = list[0].state;
		newAddress.zipCode = list[0].zipCode;
		newAddress.street = list[0].address;
		newAddress.name = list[0].firstName + " " + list[0].lastName;
		newAddress.telephone = list[0].phoneNumber;
		newAddress.found = true;
	}
	else
		newAddress.found = false;

	return newAddress;
}

Testing The Code

The project is configured to run inside the web server embedded in Visual Studio and launch the address.html page. Once the page comes up, perform the following tests.

  1. Enter a telephone number – some have been provided as example. While entering the cell information, the address associated with the phone number will be populated. The map is not refreshed until the focus is moved from the current field.
  2. Put the mouse on the label of fields with the spelling icon. The NATO phonetic spelling will be displayed if the field contains text.
  3. Press the spelling icon. This will display a dialog with the NATO phonetic spelling in both English and Spanish and allow modification of the field.
  4. Enter a valid zip code; both the state and city will be updated with the right values.

Conclusion

Reverse phone look up combined with resolution of city and state from a zip code can increase the accuracy of a customer address and reduce the time it takes a CSR to capture this information. Phonetic spelling can standardize the way information is spelled back to the customer and alleviate language related problems. These techniques combined reduce the number of bad addresses in your application database and the time CSRs spend with customers.

Download Source – 369.99 KB