Embedded Systems Software, Computer Networking and Geeky Fun

nerd1951.com

October 23, 2008

Picky-Picky C/C++ Style Conventions: postscript

Filed under: Rants, Programming — Harvey @ 6:20 pm

I goofed yesterday on one of the code examples as one reader, Michael, pointed out.  I was trying to show how to define a hardware register as a cosnt pointer to volatile data and I wrote:

static const uint8_t* volatile dataOut = 0x20000010;

which is a volatile pointer to const data.  This may be useful in some bizzarre application but it’s certainly not what I was trying to do.  Here is what I meant to do:

static volatile uint8_t* const dataOut = 0x20000010;

Fortunately, I had the code correct in the driver I’ve been working on and I hope I didn’t confuse anyone.  Actually if it had been real code instead of a web page, I’m sure the compiler would have complained the first time I tried to write to the register.  That’s one reason to have all of these qualifiers on your data.  Error messages are a great help, especially when you’ve been (trying to) code for eighteen hours.

Thanks for pointing this out Michael.

• • •
 

October 22, 2008

Picky-Picky C/C++ Style Conventions

Filed under: Rants, Programming — harvey.sugar @ 2:38 pm

By convention, at this point in time you can assume that a char in C is 8 bits, an int or a long is 32 bits and a short is 16 bits but not always. That can be a problem in embedded system programming since we often need to know the exact size of a variable. We also tend to use unsigned variables a lot and typing out unsigned gets tiresome. So you often see people inventing their own coding conventions such as ubyte, ushort, and ulong, etc. I work on code every day that may use two or three different conventions depending on who worked on the code last.

There is a standard way of expression variable sizes and whether they are signed or unsigned in the GNU and POSIX worlds. It is to use the header file stdint.h which defines a number of types of specific sizes: int8_t, uint8_t, int16_t uint16_t, int32_t, uint32_t. The stdint.h file is customized for the processor that you are working with so you can depend on the sizes. This header file also specifies things like the minimum and maximum values that can be represented by a type such as INT32_MAX, UINT32_MAX, and INT32_MIN, etc. All of this is defined in the man page for stdint.h if you are using Linux or Unix.

Sometimes you don’t really care about the exact width of a variable but you want to use a data width of some minimum size that is the most efficient for your processor. Stdint.h provides representations for that too, like: uint_fast8_t and int_fast32_t. There is also a definition for the widest types supported: int_max_t and unit_max_t. You could find out how many bytes wide these are by using sizeof(int_max_t) for example.

stddef.h is another useful header file. It defines things like NULL (though according to Stroustrup you should just use 0 for null pointers) and size_t. size_t is defined as the type returned by the sizeof() function. size_t is useful for times when you don’t really care about the size of a variable, as long as is large enough to represent the size of a data object in bytes. For example, suppose you want the binary inverse of every byte in and array:

for(size_t i = 0; i < sizeof(array); i++)
{
array[i] = ~array[i];
}

Finally, do you know what this represents?

static const uint8_t* volatile dataOut = 0x20000010;

This is a const pointer to a volatile byte at location 0×20000010. A hardware register.

You want a const pointer because the hardware registers shouldn’t be moving around. Declaring the pointer is const allows the compiler to catch you if you forget to dereference the pointer when writing to the location.

The value stored at location 0×20000010 could change on its own without the software changing it. Volatile tells the compiler not to optimize away any reads or writes to this location. For example if you only ever write to the register and never read it, the compiler might remove what seems to be unnecessary writes to this location. The optimizer might completely remove the variable from the object code. Volatile lets the compiler know that this location is special.

• • •
 

October 2, 2008

uCLinux and the Analog Device Blackfin Processor

Filed under: News, Projects, Tools, Programming — harvey.sugar @ 3:35 pm

I’ve been doing a bit of work lately on a project that uses the Analog Devices Blackfin processor. The application is very ‘net-centric so we decided to use Linux, specifically uCLinux for the operating system. Of course this means that our development tools are the GNU compiler collection and all of its related applications.

Often, one of the hardest parts of a project like this is getting Linux up and running on your target hardware. The Blackfin uCLinux web page is very well organized and has a lot of good information. I downloaded the tools and the Linux distribution from their site and had Linux up and running on a development board in a matter of hours.

One thing that is an absolute necessity is to have a JTAG interface for programming FLASH through the processor. Once you have the boot program running you can program the FLASH using commands that the bootstrap provides but you need to get the bootstrap in FLASH first. An Austrian company, Blue Technix sells a USB JTAG ICE for the Blackfin called the Ice bear. It’s about $320 (US) and since they are in Austria, it could take a couple of weeks to get one so you need to plan ahead. The ICE worked just as advertised and was critical to the success of my project. Blue Technix also has some low cost eval boards too but given the Euro/Dollar exchange rate you might do better getting something from Analog Devices or one of their distributors.

As I was saying, the hard part is often getting Linux up on your own hardware. My target had some fundamental differences from the Eval board that I started working with. Like most micro controllers, the Blackfin has a bunch of multi-function pins. It also has two UARTS. Well our design used UART0’s data out pin as a software controlled hard reset. The other fundamental difference was that we were using SPI serial FLASH instead of the parallel NAND FLASH used on the eval board.

Porting the bootstrap went pretty smoothly. After about a days work, I had the bootstrap working. But it was different with the Linux kernel. I disabled UART0 and enabled UART1 using make menuconfig. But every time the board started to boot – bam! It reset almost immediately. I knew that somewhere the function for that pin was being set up to be UART0’s output. It took about three days of searching the code but I finally found it. The I/O pin setup for both UART0 and UART1 was hard coded deep in the init code. After I fixed that, I could boot Linux using TFTP.

The next step was getting Linux to work out of the SPI FLASH. There are two approaches to this. You can create a complete compressed file system image for a RAM disk system. Then you have the boot program copy the image from FLASH to RAM, decompress it and go. This approach is very simple to configure because it’s the stock build that comes with the distribution. The downside is that you don’t have any non-volatile storage without jumping through hoops.

The second approach took me a couple of more days to get running. You build a file system for FLASH and a kernel image. There’s more to configure to get this to work. One thing you need to keep in mind is that you can’t use any loadable kernel modules to access the file system since you need the file system to load them. The nice part about this is that now you have a file system that’s almost like a disk. If you make changes to your application you can use FTP to load them on the target. You can also use the system logger to log to syslog (they actually call the log /var/log/messages).

Throughout this whole process I was able to get very good support from their web site and forums. If an answer wasn’t in the documentation wiki, I would most likely find it by searching the forums. When I posted questions to the forum, I often had an answer within an hour or two. The documentation and support were better than many commercial RTOSs and cross-compilers that I’ve used.

What really amazed me was how well the tools all work. I wrote a pretty complex application in C++ using many of the C++ library classes, Pthreads, sockets, and all kinds of system facilities. I got it all working on my desktop Linux machine. Then I just recompiled my application using the Blackfin version of the tools, loaded it on my Blackfin uCLinux system and it just worked.

The one thing though that you have to remember is that if you’re messing with the kernel then according to the GPL, anyone who buys your product is entitled to the source code. Then they’re also allowed to re-distribute that code. If you’re working with custom hardware then you’re going to be messing with the kernel. I don’t think that this requirement ever hurt sales of the Linksys 54G routers though. If the project is for a government agency, they may require full source code anyway.

• • •
 

September 30, 2008

Hacker’s food

Filed under: Tools, Geeky Fun, Rants, Programming — harvey.sugar @ 4:23 pm

Some things that help make a normal life pleasant can get to be distractions when you’re way behind schedule on a project. Nerds are legendary for ignoring these distractions when a technical challenge requires their full attention. Details like hygiene and nutrition are the first casualties in battles against bugs and deadlines. It’s really hard to be fresh smelling and perky looking when you’re in the middle of marathon systems integration problems and have been working for eighteen hours straight.

Over the last couple of weeks, I’ve really noticed a decline in my eating habits. I usually try to prepare my food from fresh unadulterated ingredients; lots of vegetables, beans, salad and grains and a good bit of meat too. But I cook from scratch and watch the carbs and fat. That is until I hit systems integration.

I started out last week with salads for lunch and home made chili for dinner. When the chili and the lettuce ran out, I switched to carry out food. I tried sticking to wholesome stuff like the local kabob place, easy on the rice and Chipotle which is quite healthy and tasty without the rice and tortillas.

Then I started eating at odd hours, late at night or very early in the morning. I switched to the diet of the legendary first generation of hackers at MIT, like Richard Stallman. I started alternating between Chinese carry out and pizza washed down with lots of Coke.

I knew I hit bottom this morning. Taco Bell is open late around here. They call the time between midnight and two AM, “The Forth Meal.” I found myself driving to the Taco Bell to get there before closing so I could get mine. A few hours later I was at McDonalds’ getting a sausage, egg, and cheese McGriddle and another Coke.

I’m lost now and I’ll admit it. I’m not eating again until I can cook something for myself. Right after a shower and a twelve hour nap.

• • •
 

September 24, 2008

It’s (almost) never the hardware

Filed under: Rants, Programming — Harvey @ 10:47 pm

We embedded systems programmers have some special challenges in our work.  One of the major problems we face is that the hardware we work with is often not fully exercised until our software exercises it.  So, it is tempting to blame the hardware when things don’t work right and the cause is not obvious.  I’ve done my share of blaming the hardware, sometimes to the point of embarrassment when it turned out I was wrong.

Over the years I’ve learned to work with the hardware engineers to solve a problem rather than point my finger.  I’ve learned the hard way to devise some low level tests to isolate a suspected hardware problem before I go bother the hardware designer.  Sometimes a ’scope or a logic analyzer are the best software debugging tools and embedded systems programmer can have.  But sometimes you run across a bizarre software problem that masquerades as an obvious hardware problem.  A couple of these kinds of bugs will change your approach to debugging forever.

Will Rogers once said “There are three kinds of men. The one that learns by reading. The few who learn by observation. The rest of them have to pee on the electric fence for themselves.”  For those who learn by reading or observation here are a couple of war stories about obvious hardware problems that weren’t.

The first story involves a very successful piece of test equipment.  If you ever worked with T1s and I told you the model number you would probably recognize it.  I worked on this product toward the end of its life cycle.  I did some of the last feature upgrades and became responsible for software maintenance on it.  This test set used a popular Intel counter-timer chip.  The same chip was used in the original IBM PC and there are incarnations of it in the system chips of personal computers to this day.  NEC was a second source for this part but for some reason this test set would not work with the NEC version of the timer chip.  There was even a note on the BOM (Bill of Materials) that only the Intel part could be used.

Well, one day, something fell through the cracks and a batch of these units were built with the NEC part.  I got a call from production test that the software was failing in these units.  So, I went over to the factory and picked up one of the failing units to look at with an ICE.  As soon as I took the unit apart, I saw the NEC timer and knew that was the problem but when I called the production manager, he insisted that I should take a second look at the software.  We used the same NEC time in several other products with no problems and it was getting harder to get the Intel part.  I was certain that it was not a software problem.  There were literally hundreds of these test sets in the field with this software, working just fine.  I couldn’t very well say no so I set up a a couple of breakpoints and ran the unit trough its paces.  As I single-stepped through the ISR for the timer for about the tenth time, I noticed that it pushed one more register on the stack than it popped off at the end.  There it was in x86 assembly code, a function that should have crashed every time no matter what timer chip was used.  I fixed the routine and the test set worked just fine with the NEC part from then on.  To this day I have no idea why the problem never showed up with the Intel timer chip but if it weren’t for the persistence of that production manager, that bug might have showed up at another time in another way.

The second story is about another very popular product from the same company.  I never worked on that product but a coworker told me about this one.  Test equipment tends to have a very long product lifetime.  This product had undergone so many upgrades that they decided that it was time for a major software rewrite.  The project went well until system testing.  It seemed that the units in the lab worked just fine but when they were buttoned up with the covers on, the software crashed.  How could the presence of the cover affect the software?  It seemed like a classic hardware problem.  Perhaps it was a problem with noise or temperature.  The mystery was that the previous software release worked just fine, with or without the covers on.  Debugging this was a nightmare.  You couldn’t hook up an ICE with the cover on the unit so they had to write test code and install it and put the cover back on.  There didn’t seem to be any rhyme or reason to the software crashes.  Meanwhile they were also pursuing another seemingly unrelated problem.  If two of the pins on the RS-232 interface were shorted together (like RTS to CTS) the software would crash as soon as the test set was powered up.  It turned out that in the start up code in the new version, someone enabled interrupts before the rest of the hardware was initialized.  Shorting the RS-232 pins together or putting the cover on the unit added just enough noise on a floating interrupt line to cause it to trigger an interrupt.  Since the interrupt vector table had not been initialized at this point, the interrupt vectored to an invalid address and crashed the system.  Yep, another classic hardware problem that turned out to be software.

So what should you take away from these stories?  Never assume the problem is hardware.  If it seems like it is hardware, work with the hardware engineer to solve the problem.  Don’t just point your finger.  Use a ’scope or a logic analyzer if you know how and if you don’t get an EE to drive for you.  Some very bizarre software bugs can disguise themselves as hardware problems.

• • •
 

September 12, 2008

Geeky Humor and the Large Hadron Collider

Filed under: News, Geeky Fun — Harvey @ 2:12 pm

The Large Hadron Collider has been in the news a lot lately as they begin the process of bringing it online. Any physics experiment on this scale is bound to inspire some geeky humor (I don’t believe that’s an oxymoron.) Here is a sampling of some of what I’ve found on the net:

First the question on the minds of many,
Has the Large Hadron Clollider Destroyed the earth yet?

Next is the LHC Rap. This is actually pretty well produced and explains what they are hoping to accomplish. When I watch this though, I can’t help but think that all those people dancing around probably have PHDs and learned more math by the time they were twelve than I’ll ever know.

And finally, you can always count on XKCD as a source of relevant (pardon the pun) nerdly humor:

I also found a funny play on words or two but they’re not for a non-X rated website. I’ll leave those to your imagination. A Google search for them is an optional exercise for the reader.

• • •
 

September 9, 2008

Open source voting machines?

Filed under: News, Rants — Harvey @ 9:38 pm

Every once in a while I go over to techdirt for some interesting reading while I’m “waiting for something to compile.” They’ve had a number of posts about e-voting and voting machines in general. One in particular caught my eye: Palm Beach County Lost 3,400 Votes; Claims Different Sequoia Scanners Count Differently. One of the issues that they raised is that it is difficult for local jurisdictions to verify the operation of voting machines because the manufacturers have been highly resistant to independent inspection of the systems.

In another article on techdirt, E-Voting Isn’t Perfect, But It Takes Less Work to Corrupt Big Elections, they explain why e-voting makes it so much simpler to rig elections. The article references a paper, Voting System Risk Assessment via Computational Complexity Analysis by Rice computer scientist Dan Wallach. Being the nerd that I am, I downloaded the paper and read it. It’s pretty scary if you believe in democracy.

I’m sure that the technical community can do better than this. If we don’t want companies like Diebold and Premier counting our votes using flawed equipment and security features, I think we need to do something about it ourselves. I think we should form an organization to develop an open source voting machine. I’m sure there are some very talented computer security researchers out there who could design procedures that would make vote tampering nearly impossible. Just look at OpenSSL which is used for thousands of financial transactions every day. I also think that the open source community is better at developing reliable software  than for-profit enterprises that are more interested in minimizing cost than maximizing value.

An open source voting machine would have the following advantages:

  • The complete design, hardware, software, procedures, would be open for inspection by any government body or independent panel.
  • The project could attract the best and brightest in computer and systems security to ensure state of the art fraud prevention.
  • The project would also attract top talent in hardware and software development.
  • Local government agencies could have competitive bidding for the manufacture of the voting machines. The machines could even be made available to budding democracies in other countries.
  • An open source community would be more willing to admit to flaws and mistakes and correct them.
  • If fraud is suspected the open source community would be available to analyze the evidence.

I don’t know if this is a crackpot idea or a chance for us tehies to help preserve democracy. I don’t know, there may already be a project like this going. I would be interested in what others think of this idea and if anyone might even want to work on a project like this.

• • •
 

August 6, 2008

The most amazing thing

Filed under: Tools, Rants — harvey.sugar @ 4:57 pm

I keep both a Windoze machine and an Ubuntu Linux machine on my desk.  There are a few applications that I need to use for embedded development that just aren’t available on Linux so I need to keep the Windoze machine around.  It’s actually been months since I’ve used it though.  I use Ubuntu exclusively for office stuff, web surfing, email, and entertainment like listening to music or watching DVDs.  (I have 4 computers and no working TV).

When I first turn on the computer in the morning there is often a notification for software updates.  These usually go pretty fast so I let the computer update itself while I fix my coffee and oatmeal.

A few months ago, the most amazing thing happened.  There was a notification that an entirely new revision of Ubuntu was available for download.  Being cautious, I deferred doing that update until after work.  I spent an hour or so backing up some critical files to the computer that I use as a CVS and Subversion server.  Then I started the update which only took about half an hour.  A lot of software, including the Linux Kernel was changed.  When I rebooted the computer, I was prepared for the worst but everything just worked!  Some applications were even a little faster!

Isn’t that the most amazing thing?

• • •
 

June 11, 2008

Adivce to a new software engineer

Filed under: Rants — Harvey @ 7:57 pm

My nephew just graduated with a degree in computer science and I was thinking about what advice I would give him to help insure his success.  You can measure my experience as a software engineer in decades and it’s not easy to remain a respected and successful programmer for that long.  Since this is the time of year when many new computer science graduates start out their careers, I thought I’d share my thoughts.

First and foremost, like all businesses, the computer business is a people business. Work on your people skills. You don’t have to be a slick marketing type but learn to listen to people. Stay in touch with people who you connect with, especially people you admire; teachers, managers, colleagues, and classmates.

Learn to write well. At some point, coding is no longer as challenging and creative. The most creative work is in the design, specification and architecture of a system. Much of this needs to be communicated to others using the written word. Unless you took a double major in English, you haven’t really learned to write well in school. Like coding, the best way to learn to write is to write. There are often opportunities on and off the job to write. I’ve volunteered to write user manuals, advertising copy, news letters, and white papers over the years. This blog is a way for me to work on my writing.

Never stop learning. Take on difficult assignments – not easy ones that you already know how to do. Read articles on the ‘net and technical books about subjects that interest you.  Play with new technologies on your own time so that when that opportunity comes along to do something new, you already know about the technology. When I first started out as a software engineer, C, C++, the Internet Protocols, and personal computers did not exist. Unix was still pretty much in the lab. Since then I have mastered these technologies and have become an expert in some. Nearly all of this was accomplished through self study.

Finally, take on every assignment as though it was the most important thing you have ever worked on. Do a better job than what is expected and don’t settle for doing anything just good enough. You will be more satisfied with your job and will have more confidence in your product. The quality of your work may not always be appreciated but you’ll know you did your best and that is satisfaction enough.

Congratulations, welcome to the field, and good luck!

• • •
 

June 7, 2008

Encapsulating Shared Resource Access In C++

Filed under: Programming — Harvey @ 5:22 pm

One of the most common uses of threads in embedded systems is to serialize usage of shared resource such as data, I/O ports, sockets, or files. A typical implementation is to use a queue and a thread. Threads accessing the shared resource place requests or operations on the queue. A thread waits on the queue, dequeuing requests and performing the operations on the shared resource. Structurally, it looks something like this:

Usually it’s best to hide all of this complexity from the resource users. I usually place the thread, the queue and their related components in a single class. The resource user sees only one public function; queueRequest(Request&). This helps insure that users only access the resource through the access queue/thread. The name of the function reminds the user that the request may not be executed immediately. Now let’s look at an implementation in C++ using POSIX threads:

First the class declaration:

class SharedResource
{
public:
    SharedReasouce(Resource& resource);
    ~SharedResource();

    queueRequest(Request&);

private:
    static void* accessThread(void* params);

    Resource&             _resource;
    std::queue<request&>  _requestQueue;
    pthread_t             _accessThreadId;
    pthread_cond_t     _queueSignal;
    pthread_mutex_t       _queueMutex;  // Required to use pthread condition code
};

We pass a reference to the resource to the constructor. This way if we have several resources of the same type, we can create an object using this class for each of the resources.

The thread function must be static in order to use it with the POSIX create_thread function. Thread functions take a single parameter, a void pointer. We will use this pointer to pass the this pointer to the thread.

The queueSignal condition code is used to signal accessThread when a request has been placed on the queue. Pthreads requires that you use a mutex to signal a condition code and this is what queueMutex is used for.

Now let’s look at the constructor:

SharedResource::SharedResource(Resource& resource)
{
    _resource = resource;
    pthread_mutex_init(&_queueMutex, 0);
    pthread_cond_init(&_queueSignal, 0);
    pthread_create(&_accessThreadId, 0, SharedResource::accessThread, this);
}

The constructor makes a copy of the resource reference and creates the pthread condition code and mutex.  Then the access thread is created.  This starts execution of the function  SharedResources::accessThread.  A pointer of this instance of class SharedResource, the this pointer, is passed to the thread function as a parameter.

The queueRequest function is streight-forward.  It simply places a request reference in the queue then signals the accessThread that a request is in the queue:

void SharedResource::queueRequest(Request& request)
{
    _commandQueue.push(request);
    pthread_cond_signal(&_queueSignal);
}

Finally let’s take a look at the thread itself:

void* SharedResource::accessThread(void* param)
{
    // Convert the void* parameter back to a SharedResource reference
    SharedResource& sharedResource = reinterpret_cast<SharedResource&>(param);
    while(1)
    {
        // Wait on the request queue
        pthread_mutex_lock(&sharedResource._queueMutex);
        pthread_cond_wait(&sharedResource._queueSignal,
            &sharedResource._queueMutex);

        // Dequeue any requests and process them
        while(sharedResource._requestQueue.size() > 0)
        {
            request = sharedResource._requestQueue.front();
            sharedResource._requestQueue.pop();

            // process the request on resource
        }
    }
}

A couple of things to note.  In an actual program, you would check the return codes of each of the function calls in the constructor that allocate resources.  If one of these functions fails you need to clean up any allocated resources and probably throw an exception.

• • •
 
Next Page »