The IronPython 2 Parser

From IronPython Cookbook

The example of parsing expressions is now out of date. In the meantime Dino has provided this example of using the IronPython 2 parser with 2.0 beta 4 and beta 5:

import clr
clr.AddReference('IronPython')
clr.AddReference('Microsoft.Scripting')
clr.AddReference('Microsoft.Scripting.Core')
from IronPython.Compiler import Parser

# beta 4
from Microsoft.Scripting.Hosting import HostingHelpers
from Microsoft.Scripting.Hosting import ScriptRuntime
from System.Scripting import ErrorSink
from Microsoft.Scripting.Compilers import CompilerContext
py = ScriptRuntime.Create().GetEngine('py')

# beta 5
#from Microsoft.Scripting import ErrorSink
#from Microsoft.Scripting.Runtime import CompilerContext
#from Microsoft.Scripting.Hosting.Providers import HostingHelpers
#from IronPython.Hosting import Python
#py = Python.CreateEngine() # beta 5 and beyond

src = HostingHelpers.GetSourceUnit(py.CreateScriptSourceFromString('print "hello"'))
pylc = HostingHelpers.GetLanguageContext(py)


p = Parser.CreateParser(CompilerContext(src, pylc.GetCompilerOptions(), ErrorSink.Default), pylc.Options)
ast = p.ParseFile(True)

print ast.Body

---

This is an example of using the IronPython 2 parser to parse Python expressions and extract the variable names used as a list of strings.

It works with IronPython 2.0a3 (including Silverlight 1.1 Alpha Refresh), but the parsing API is not stable so it may not work unmodified with later versions.

This code requires a subclass of an IronPython core class called the 'PythonWalker'. This is used for walking the IronPython AST. It has a method called 'Walk' with an overload for each node type. This could be subclassed from IronPython code (which would then need to dispatch on the different types of node passed in).

Instead I used a small piece of C# to collect the 'Names' it encounters:

using System;
using System.Collections;
using System.Collections.Generic;
using IronPython.Compiler.Ast;
using Microsoft.Scripting;

namespace ExpressionWalker
{
    public class ExpressionWalker : PythonWalker
    {
        public List<string> names = new List<string>();

        public override bool Walk(NameExpression node)
        {
            names.Add(SymbolTable.IdToString(node.Name));
            return true;    // means walk this node, not important for a Name node.
        }
    }
}


This can then be used from IronPython code in the following way:

import clr
clr.AddReference('ExpressionWalker')
clr.AddReference('IronPython')
clr.AddReference('Microsoft.Scripting')

from IronPython import PythonEngineOptions
from IronPython.Hosting import PythonEngine
from IronPython.Compiler import Parser
from Microsoft.Scripting import CompilerContext, SourceCodeUnit

from ExpressionWalker import ExpressionWalker

expression = "A1 * 3 + B4"
pe = PythonEngine.CurrentEngine

s = SourceCodeUnit(pe, expression)
c = CompilerContext(s)
p = Parser.CreateParser(c, PythonEngineOptions())

e = p.ParseExpression()

w = ExpressionWalker()
e.Walk(w)

print list(w.names)

When run, it prints:

['A1', 'B4']

These are the variable names used in the expression passed into the SourceCodeUnit at the start.

The pure Python equivalent of ExpressionWalker is:

>>> from IronPython.Compiler.Ast import PythonWalker
>>> from IronPython.Compiler.Ast import NameExpression
>>> from Microsoft.Scripting import SymbolTable
>>> class ExpressionWalker(PythonWalker):
...     def Walk(self, node):
...         if type(node) == NameExpression:
...             print SymbolTable.IdToString(node.Name)
...         return True
...


Back to Contents.

TOOLBOX
LANGUAGES