[Pyparsing] parsing grammars with suffixes (array indexing, attributes)

[Andrea Censi]

Hi,

I have been using PyParsing for some major projects so far, but I still
don't understand how to properly parse some specific types of recursive
grammars, for which the function operatorPrecedence does not help.

In particular, take Python's syntax for indexing using square brackets.
These are some examples:

array[0]
array[0][1]
array[0][1][2]

I understand why the following solution does not work:

value = Forward()
value_ref = Word(alphas)
value_index = value + "[" + index + "]"
value_ ... = ...
....
value << (value_ref ^ value_index ^ value_... ^ ...)

(This is a very simplified version of what I am dealing with, in which I
have about a dozen possible expressions for value).

I understand why something like the above gives an infinite recursion
exception. The problem is clear. Yet I don't see the solution.

Another example that appears also in python is dealing with attributes:

object
object.attribute
object.attribute.attribute2
(expression ...).attribute

I suspect that this is a general problem that people have.

Any hints?

thanks,
A.

[Paul McGuire]

Andrea -

While the convential BNF approach is usually to use left-recursion for these
types of expressions, pyparsing does not do well with left-recursion.
Instead, you are better off using ZeroOrMore/OneOrMore for the repetition:

indexed_reference = identifier + ZeroOrMore('[' + integer_expr + ']')

This actually can be more complicated, since the integer_expr could itself
be an indexed_reference, if one array contains indexes into the other:

array1[array2[0]]

You may also want to account for slice notation (assuming this is Python
that you are parsing):

slice_expr = '[' + integer_expr + Optional(':' + integer_expr +
Optional(':' + integer_expr)) + ']'
indexed_reference = identifier + ZeroOrMore(slice_expr)

This has been a busy week, so I've not had more chance to delve into this in
detail. I know that I've written a pyparsing generator (in pyparsing) that
reads the Python BNF and creates a pyparsing parser for it. I'll have to
dust this off and see how this is done in that code (you can also look for
yourself, I'm pretty sure it is in the shipped/online examples).

-- Paul


-----Original Message-----
From: Andrea Censi [mailto:***@mit.edu]
Sent: Saturday, July 02, 2016 6:17 PM
To: pyparsing-***@lists.sourceforge.net
Subject: [Pyparsing] parsing grammars with suffixes (array indexing,
attributes)

Hi,

I have been using PyParsing for some major projects so far, but I still
don't understand how to properly parse some specific types of recursive
grammars, for which the function operatorPrecedence does not help.

In particular, take Python's syntax for indexing using square brackets.
These are some examples:

array[0]
array[0][1]
array[0][1][2]

I understand why the following solution does not work:

value = Forward()
value_ref = Word(alphas)
value_index = value + "[" + index + "]"
value_ ... = ...
....
value << (value_ref ^ value_index ^ value_... ^ ...)

(This is a very simplified version of what I am dealing with, in which I
have about a dozen possible expressions for value).

I understand why something like the above gives an infinite recursion
exception. The problem is clear. Yet I don't see the solution.

Another example that appears also in python is dealing with attributes:

object
object.attribute
object.attribute.attribute2
(expression ...).attribute

I suspect that this is a general problem that people have.

Any hints?

thanks,
A.
----------------------------------------------------------------------------
--
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Pyparsing-users mailing list
Pyparsing-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pyparsing-users


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus