FORTH DEVELOPMENT ENVIRONMENT

I believe that Forth like any other programming language benefits from a well organised project development environment.

This article is a work in progress because it’s using Open Software on the PC allowing me to continually improve my designs.

Trqaditional Forth

Traditional Forth uses a serial terminal to interactively command (in real time) and upload source to the MCU. The Forth I use with Cortex-M is a traditional but modern Forth.

Traditional Forth serial terminal use has a few variations not limited to my description below because anything is possible with Forth.

  1. A terminal such as Picocom, Minicom even Hyperterm in Windows is used to interactively command in real time and to upload source to the MCU making use of the ASCII serial upload capability of the terminal or a ‘helper’ program called by the terminal for uploading. In these cases the source code comments are read but rejected by the on-mcu Forth compiler and End Of Line delays are inserted by the terminal (where supported) to prevent the on-mcu Forth compiler choking on long or complex source. There is usually no handshaking of any kind to use as a alternative to the EOL delays and speeds are often 9600 Baud so that errors and warnings from the on-mcu Forth compiler can be read by the programmer as they occur.

Advantages

  • simple

Disadvantages

  • unbearably slow

  1. A purpose made terminal with on board smarts and file upload capability such as eForthcom is used. This special terminal is made especially to suit Forth, and strips the comments from the source, recognizes errors, highlights them in color and stops the upload completely at the error. eForthcom works by waiting for the “OK.” return from the on-MCU Forth compiler giving the fastest possible upload speed without EOL delays or traditional hardware handshaking.

Advantages

  • quick and easy

  • very fast

  • limited to a maximum speed of 115,200 Baud

  • configurable error detection for a predefined choice of different Forths

  • coloured error messages

  • customisable error handling

  • halts on errors

  • written in Forth and bugfree

  • actively maintained by the developer Manfred Mahlow

Disadvantages

  • Linux only binary distribution (I use FreeBSD)

My Development System

Gnu Screen is the key component of my system and nothing else I’m aware of comes close. Not Tmux, not CU, nothing.

Essential Development Capability

Only Gnu Screen satisfies all my requirements:

Requirement

Because ?

a serial terminal

talk Forth to the MCU

hardware handshaking

maximise the speed of source code uploads to the MCU

controllable logging

acquire MCU code; automated cloning of MCU Flash image

remote named terminal sessions

automated script control of the terminal

remote commands

makefile control of source code uploads to MCU

remote uploads

makefile control of source code uploads to MCU

at least one sub process

Forth on-chip compiler error message detection (via SED) and terminal bell alert on errors

Key Aspects of the development system

  1. Interactively command the on-mcu Forth in the gnu screen window.

  2. upload source to the mcu when I click my editor ‘make button’. It does this by opening a remote connection to gnu screen and uploading commands and source via a Makefile. I do all this from the editor window but I see the upload as it occurs in the gnu screen window … sort off, because at 460800 baud with hardware handshaking its hard to read anything as it flashes up the screen.

  3. all comments are stripped from the source by SED before being uploaded in b. above

  4. Any errors from the on-mcu Forth compiler are highlighted in RED in gnu screen using a sub-process involving configurable SED keywords and ANSII escape codes inserted into the RX stream by the MCU. Warnings are coloured BLUE. Warnings and errors RING the gnu screen terminal bell as they occur. This is required because the uploads are to0 fast to read.

  5. Stop the upload at the error if desired. This can be done with a toggle switch at the development hardware. I don’t use this nowadays as color errors and the beep of the terminal bell are all I need. Gnu screen has a long history buffer and I can easily scroll back to look for the colored error after hearing a beep.

  6. Suck the entire Dictionary from the MCU in IntelHex and build a complete binary image of everything including user Words from a project. This image can then be flashed to another mcu which is then a 100% clone.

Personally I find traditional Forth such as 1. above far,far,far too slow and I wouldn’t blame anyone who tries Forth in this way from thinking it was a load of ancient crap compared to using arm.none.eabi and GDB via SWD on a Cortex-M.

I developed my system until it was faster than arm.none.eabi and GDB via SWD on a Cortex-M using the usual write code, try, fix, repeat development cycle.

My system is far more than just gnu screen above as it involves:

  1. a project builder

  2. integrated Fossil SCM with webserver

  3. integrated configurable CMSIS-SVD memory map and bitfield generator for any STM32 Cortex-M mcu. This means I can easily write Forth code as I have every peripheral memory mapped automatically. I never have to refer to the documentation for memory locations or register bitfields. Doing this manually is tedious, error prone and incredibly slow. No one should ever have to do this and I’m amazed by Forth people who still do this manually on Cortex-M.

Overall Operation

  1. Create project directory

  2. Cd to project directory and type “project-builder”, which fills the directory with project templates and stubs

  3. Edit project source code

  4. Run Make which:

    1. Resets the target MCU

    2. Strips all comments from the Forth source

    3. Uploads the Forth source to the target at 460800 Baud with hardware handshaking

  5. Test target

  6. If correct target operation observed, go to 7. else go to to 3. and repeat

  7. Finished. Run “clone.sh” and create a MD5 named executable project binary for the target.

PROJECT BUILDER

Any Forth project is going to need a well structured directory containing quite a few files and doing this by hand for every project is error prone and incredibly tedious, in fact it’s just not reasonable.

In my case I built a “project builder” shell script (currently about 800 lines) which generates project stubs automatically. This system is somewhat similar to CrystalScad

Basically I just create a directory named for the new project, then in that directory type the name of my script and everything is auto created in about 0.1 seconds.

This picture shows the directory contents of a brand new, automatically pre-populated project

_images/project-builder-files-created.jpg

Project Builder Highlights

  • Makefile creation

  • Code Versioning System: set up and running, with the initial project files already committed. I use Fossil SCM for this.

  • Schematic Capture: with a various templates ready for me to begin testing and documenting. Usually I start with a STM32F0 Discovery Board, so that’s my main schematic template. gEDA is what I use.

  • Directories: ready to receive files as I create them, i.e. pdf, flowchart, pics, text, misc

  • A library symlink to my versioned STM32F Mecrisp-Stellaris library so I can pull them in as needed

  • Py.Sphinx documentation system: (py.sphinx is used to create this site) ready to start adding documents.

  • Editor: which will open with several files ready to edit, pre-created and pre-named by the project builder shell script.

  • Serial Terminal

  • Svd2forth files to suit the MCU. These are used to generate Memory Mapped Register words, preloaded before the actual project files and controlled by a Makefile.

  • Utility applications such as cloning the project source into a executable target binary.

  • Finally, a z-WARNING-DESTROY-test-project.fossil.sh shell script that will remove every project file, the Fossil repository and finally the project directory itself. This has a “are you sure” confirmation before proceeding. Once a project is underway and I’m happy with it, I may manually delete the z-WARNING-DESTROY-test-project.fossil.sh shell script just to be on the safe side. It’s saved in the Fossil Code Repo anyway if I need it later.

The project builder shell script also refuses to dump all these files into a non-empty directory.

It’s all done with one SH script, no Java, no Eclipse, no delay.

In fact, the above system is perfectly responsive on a Raspberry Pi V3 with no delays. Try running the free Atollic TrueStudio for STM32 on a RPI ? I bet it takes a week just to load, and that’s not including CubeMX.

This picture shows the auto opened editor files

_images/project-builder-gvim.jpg

SERIAL TERMINAL

https://www.gnu.org/software/screen/ showing coloured error messages from the MCU on chip Forth compiler.

This picture shows various contrived error and warning messages in colour

_images/mecrisp-stellaris-coloured-errors1.jpg

I edit Forth Code in a GUI editor then click on the editor “make’ icon to test the code on the target. This strips all comments from the source,and uploads the code at 460800 Baud with RTS handshaking to the target MCU which then runs the code.

The PC ‘bell’ also emits a loud “beep” when ever a error is reported on the terminal. It does this by parsing all received data thru Sed where error keywords ring the terminal bell.

This method is the fasted I’ve found to date. When a large and complex Word is encountered the GNU Screen Window freezes for a instant while the target MCU compiles it then resumes the upload scrolling code on the terminal screen faster than I can read it.

For years I used a few dedicated Serial Terminals, the best being e4thcom but only Gnu Screen has allowed the freedom to implement new features as they occur to me.

Features

  1. Allows the editor to perform uploads with a ‘Make’ button click

  2. Colour error highlighting

  3. Speed of 460800 Baud to the target

  4. PC bell beeps on configurable target errors

  5. Comments automatically removed before upload to the target.

Frankly if one can’t detect code errors automatically during a fast source upload, Mecrisp-Stellaris is going to be a very disappointing experience ihmo. I believe watching a large Forth program upload slowly just so errors can be observed is error prone, painfully slow and outdated compared to modern C development systems.

In my experience this system is faster than using C and GDB on the same Intel i7 workstation.

CrystalScad

https://github.com/jglauche/CrystalScad

CrystalSCAD comes with a generator that generates project stubs automatically for you. Run this command from a terminal in the directory that you want to create a project: # crystalgen new [my_project_name]

Change [my_project_name] to the name of your project

A project named “my_project” will create those files and directories:

my_project/my_project.rb - the controller
my_project/lib/assemblies - for putting together assemblies of individual parts
my_project/lib/electronics - put electronics here
my_project/lib/hardware - put hardware parts in here
my_project/lib/printed - put parts that you want to print in here
my_project/lib/lasercut - put sheets that need to be cut (by laser or other) in here
my_project/lib/assemblies/my_project_assembly.rb - dummy assembly
my_project/lib/printed/testcube.rb - dummy printed part
my_project/my_project.observr - observer file

Open up the controller (here my_project/my_project.rb ) in the text editor of your choice. It contains the information on how to start the observer.