Objectives

1. I want to learn the Python language and at the same time build a software.
2. As an assembler for the 68000 Simulator (My final project for Degree in Electrical Engr. at UiTM, Malaysia)
3. Use the regular expression.
4. Make simple, short and readable code in Pythonic way.

Brief History

I discover the Python language after introduced in my favorite Java book, Thinking in Java ¹. I start study Python early Dec 2003 and start this project at middle of Dec 2003. The Python bite me! After create the classes, I spent a lot of time to refactor ⁵ it and make it Pythonic way with help from Python Cookbook ², Python's projects in SourceForge, etc. Long live Python Community!

Features

1. Can assemble most of the 68000 instructions.
2. Recognize most of the addressing modes.
3. The output can be used by my 68000_Simulator for simulation.
4. Use regular expressions for operand parser.
5. You can use, modify, etc. the source code for this project. Please keep the credit, and let me know :-)

Comparison to Flight68k Cross-Assembler. Please refer to examples in directory examples and listing files in directory test/lst.

1. Detail error reporting. Refer test/witherrs.lst and test/witherrs2.lst.

• hi68kASM:

  # Assembler used: Helmi's Assembler
  (truncate here...)
  
  ERRORS INFO:
  ============
  **** ERROR, File "D:\Helmi's Documents\Project Python\assembler\test\with_errs.asm", line 7
  Operand 2 not Immediate:     link    a3,^$12
  
  **** ERROR, File "D:\Helmi's Documents\Project Python\assembler\test\with_errs.asm", line 9
  Size not B: m    nbcd.^W    d2
  
  (truncate here...)
  
  Assembly FAILED
  13 errors
  0 warnings
  

• Flight68k Cross-Assembler:

  witherrs.asm: **** ERROR at line 7       Bad addressing mode
      link    a3,$12^
  
  witherrs.asm: **** ERROR at line 9       Illegal forcing
  m    nbcd.W    ^d2
  
  (truncate here...)
  
  Pass 1 complete
  13 errors
  0 warnings
  

2. Display symbol table. Refer test/withasm.lst. e.g:

   SYMBOL TABLE:
   
   Symbol    Value
   ------------------
   max       00400404
   

Limitations

1. Cannot evaluate value. e.g:

   move    32+2-1,d0
   

2. Not implemented:

• Instruction Movem
• Addressing mode SP, PC with Displacement and PC with Index
• Output for machine code (.bin)
• Report for Warnings.

3. Cannot recognize symbol in addressing mode Adresss Register Indirect with Displacement and Address Register Indirect with Index. e.g:

   sym1    nop
           move    #$1234,sym1(a0)          ;error
           move    #$1234,sym1(a0,d3)       ;error
   

Command line

Write this command in a console, e.g. cmd.exe in WindowsXP. Assume that the assembler.py in current working directory:

python assembler.py example.asm

Then, the console will display the expected_output. To create a listing file, add flag -f in the command. e.g:

python assembler.py -f example.asm

A file, example.lst will be created. You can use this to simulate the operation in my 68000_Simulator.

Problems??? Try replace python to C:\Python23\python.exe (path where you installed the Python)

See snapshots of a typical usage in my platform, output_cmd, debug_cmd

In SciTE

This is my favorite source editor :-). Open the SciTE ³ and choose menu Options>Open asm.properties. Add the following lines in the properties and save:

command.name.1.*.asm=assembler
command.1.*.asm=c:\Python23\python ..\hi68k\assembler.py $(FileNameExt)
command.1.subsystem.*.asm=2

command.name.2.*.asm=assembler (create .lst)
command.2.*.asm=c:\Python23\python ..\hi68k\assembler.py -f $(FileNameExt)
command.2.subsystem.*.asm=2

Open the example.asm. Choose Tools>assembler. assembler.py will process the file and display the expected output at Output (choose menu View>Output or press F8).

Two new submenus in Tools will appear every time you open a file (.asm). Note at line command.1.*.asm. The format is command.1.*.asm= <Where is the Python> <Where is the assembler> <the filename>. In SciTE, $(FileNameExt) is the name for current file opened.

See snapshots of a typical usage in my platform, output, debug

In hi68kEditor

No documentation yet!

Expected output

\/ Begin process example.asm
Commencing first pass ... OK

# Assembler used: Helmi's Assembler
# Source: File "example.asm"
# Created on: Sun 29 Feb 2004, 01:01 AM


LINE  LOCATION OBJECT CODE          SOURCE TEXT
----- -------- -------------------- ------------------------------
    1 00400400                          org       $400400
    2
    3 00400400 303C1234                 move.w    #$1234,d0
    4 00400404 06000034                 addi.b    #$34,d0
    5 00400408 4E71                     nop
    6
    7                                   end

Assembly complete
Bytes filed: 10
0 errors
0 warnings
/\ End process example.asm

What I like about Python!

Part of Python that I found elegant, concise, and readable. I show part of the source code from hi68kASM that I used it.

1. Data structure.

• tuple. Refer operand.py:

  opList = (DnDirect, Abs, Imm, AnDirect, AnInd, AnIndPost,
            AnIndPre, AnIndDisp, AnIndIndex, PCDisp, PCIndex)
  

• list. Refer method Assembler.process() in assembler.py:

  self.relocList = []
  

• dictionary. Refer method Assembler.__init__() in assembler.py:

  self.symbolTable = {}
  

tuple, list and dictionary are built in data types. For Java the programmers, the analogy are tuple = array, list = ArrayList/Vector, dictionary = HashMap.

2. List comprehension.

-filter. Refer class DataMode, in operand.py:

modes = [op for op in opList if op is not AnDirect]

-map. Refer method Directive.dc(), in assembler.py:

val = [("%08X" % intPreSign(op))[-oft:] for op in self.ops.split(',')]

-map and filter. Refer method _Opcode.extendOp(), in instruction.py:

ext = [op.val for op in (self.op1, self.op2) if hasattr(op, "val")]

A concise way to create a list by map, list, or both the values in the list/iterable. See also Python Tutorial, 5.1.4.

3. Doctest ⁶.

As example of usage and can run as a test script. Refer, assembler.py:

class Directive:
    
    def __init__(self, asm, sym, size, ops):
        self.asm=asm; self.sym=sym; self.size=size; self.ops=ops
        
    def dc(self):
        """Define constant.
        
        >>> asm = Assembler()
        >>> ops = '$12,$13'
        >>> Directive(asm, '', 'b', ops).dc()
        '1213'
        >>> ops = '$12,$133'
        >>> Directive(asm, '', 'w', ops).dc()
        '00120133'
        >>> Directive(asm, '', 'l', ops).dc()
        '0000001200000133'
        """
        oft = {'b':2, 'l':8}.get(self.size, 4)  # Maybe size is None
        val = [("%08X" % intPreSign(op))[-oft:] for op in self.ops.split(',')]
        self.asm.incPc += oft*len(val)//2
        return "".join(val)

4. Dynamic.

• return type. Refer method Assembler.tokenize() in assembler.py:

  def tokenize(self, line, asList=False):
      ... (truncate..)
      res = sym, mne, ops
      if not asList:  # Return as dictionary.
          res = dict(zip(('sym', 'mne', 'ops'), res))
      return res      # Return as list.
  

• import. Refer function my_import() in test_assembler.py:

  mod = __import__(name)
  

• Getting object references. Refer method Assembler.assemble() in assembler.py:

  if hasattr(Directive, mne) and mne != '__init__':
      return getattr(Directive(self, sym, size, ops), mne)()
  

5. New style class.

• __getitem__. Refer class ListFile in listfile.py:

  def __getitem__(self, key):
      """
      >>> import assembler
      >>> lstFile = ListFile(assembler.Assembler(), 'Test')
      >>> lstFile.listTable.append('line 1')
      >>> print lstFile[0]
      line 1
      """
      try: 
          if self.errNo == 0:
              return self.listTable[key]
          return self.listErr[key]
      except: return ''
  

6. Regular Expressions.

• Refer module operand.py:

  reReg = '(?P<reg>[0-7])'
  class DnDirect:    
      re1 = 'd%s' % reReg        
      def __init__(self, reg=None):
          """        
          >>> f = re.compile(DnDirect.re1).match("d3")
          >>> dn = DnDirect(**f.groupdict())
          >>> dn.reg
          3
          """
          self.reg = int(reg)
  

Match a string to the regular expression. Then, get the found group as dict, and pass it as keyword to initialize a class.

Conclusion

Please refer API for the structure of classes, modules, etc for the 68000 Assembler.

instruction set

Design as classes not functions. Easy to extend, e.g. add method to do the simulation.

operand parser

I had build a simple operand parser using regular expression. Recommend to refer EASy68K ⁴ for complete operand parser. You can get the complete C code. I am not good in C and not understand the function for the operand parser(that why I design my own parser!).

unit test

Build unit test before the code!. Make you confidence to refactor ⁵ code for better design. Provide doctest ⁶ also is a good practice.

design pattern

Improve the design. Help for maintenance. I always keep refering to Patterns_in_Python and Thinking_in_Python.