[Pyparsing] Parsing Stanford Polygon File Format (PLY)

[E Young]

Dear All,

first off, I would like to thank the developer(s) for making such a great
tool. It's such an elegant piece of work!

Though, I'm happy if somebody can provide some help. I'm trying to
implement a Standford polygon file parser (
http://paulbourke.net/dataformats/ply/), but I'm having trouble getting it
to parse the file into the correct data structure, because parts of the
file depend on depend on declarations earlier in the same file.

For those not in the know, the PLY file format has a header and a body, as
seen in the example posted on the aforementioned link. The header declares
how the body is supposed to be parsed and the body consists only of numbers
(either textual or binary). The declarations dictate the order of the data
in the body. My attempt at a context free grammar is as follows:

ply_grammar ::= header body
header ::= "ply" declaration+ "end_header"
declaration ::= format | element | property
format ::= "format" format_type NUMBER
element ::= "element" element_type NUMBER
property ::= ("property" property_type IDENT) | ("property" "list"
property_type property_type IDENT)
format_type ::= "vertex" | "face" | "edge" | IDENT
property_type ::= "char" | "uchar" | "short" | "ushort" | "int" | "uint" |
"float" | "double"
body ::= statement+
statement ::= NUMBER+

The code is available at:
https://gist.github.com/anonymous/f7fee82634ba224e25e9022ec2c3c890

So far, I managed to parse the file header such that each declared element
has nested property declarations, but I'm unable to tell pyparsing how to
parse the body data accordingly. In the end, I would like to have instances
of numpy.ndarray or array.array for each element declared in the header.
Any help is greatly appreciated!

Best,
E

[p***@austin.rr.com]

Post by E Young
first off, I would like to thank the developer(s) for making such a great
tool. It's such an elegant piece of work!

Thank you - flattery is always a good start! I'm glad pyparsing has been helpful for you, and that you find it to your liking.

Kudos on your posted parser - starting with posting your BNF! So many people skip this step, and then get mired down in distracting details, and find they have overlooked or skipped significant parts. I'm also glad to see that you are using the new pyparsing_common expressions. I had some mixed feelings about including them, and your sample code shows them being put to good use.

Post by E Young
Though, I'm happy if somebody can provide some help. I'm trying to
implement a Standford polygon file parser (
http://paulbourke.net/dataformats/ply/), but I'm having trouble getting it
to parse the file into the correct data structure, because parts of the
file depend on depend on declarations earlier in the same file.

This happens often enough, and there are several examples of such adaptive parsers that I have posted over the years. The main concept is to define a Forward for the part of the parser that needs to be adaptive, and then to use a parse action attached to the part with the format description to create the flexible part of the parser, and to then insert it into the placeholder Forward using the '<<' operator.

I touched up your parser from the gist that you posted, and have added some sample code that will parse the test cube example. You can find it here: https://gist.github.com/anonymous/4498de5ce91da8cf4292bad408076b97 I think you'll be able to take it from there. (I also applied some stylistic changes - removing explicit .setResultsName calls in favor of implicit callable notation, changing CaselessLiteral to CaselessKeyword, a Group added here or there for better structure and results name assignment. Feel free to keep any, all, or none of these changes.)

Please post a link to your finished project to this list, or the Pyparsing Facebook page.

Regards,
-- Paul McGuire

[E Young]

Hi all,

it took a while, but I managed to complete an initial working version of
the PLY parser thanks to Paul's suggestions. The self-contained example you
can find here:
https://gist.github.com/youngec/76c5b552ab5bc04b43d1bfa9d3fa0a78. Its only
dependency is the python package attrs (https://pypi.org/pypi/attrs),
because it makes python classes a bit easier to manage. You're free to use
that parser as you wish.

The parser is part of a larger project at
https://github.com/youngec/rootspace/tree/develop, and the respective unit
tests are at
https://github.com/youngec/rootspace/blob/develop/src/rootspace/tests/test_parsers.py
.

Theres one big caveat to the parser though, and I'm hoping you can provide
some guidance: Currently, it only understands ASCII-based PLY files, but
doesn't understand the two much more common binary formats (little- and
big-endian). Can pyparsing handle binary data?

Thanks so much!

Best,
Ellie

Post by p***@austin.rr.com

Post by E Young
first off, I would like to thank the developer(s) for making such a great
tool. It's such an elegant piece of work!

Thank you - flattery is always a good start! I'm glad pyparsing has been
helpful for you, and that you find it to your liking.
Kudos on your posted parser - starting with posting your BNF! So many
people skip this step, and then get mired down in distracting details, and
find they have overlooked or skipped significant parts. I'm also glad to
see that you are using the new pyparsing_common expressions. I had some
mixed feelings about including them, and your sample code shows them being
put to good use.

Post by E Young
Though, I'm happy if somebody can provide some help. I'm trying to
implement a Standford polygon file parser (
http://paulbourke.net/dataformats/ply/), but I'm having trouble getting

it

Post by E Young
to parse the file into the correct data structure, because parts of the
file depend on depend on declarations earlier in the same file.

This happens often enough, and there are several examples of such adaptive
parsers that I have posted over the years. The main concept is to define a
Forward for the part of the parser that needs to be adaptive, and then to
use a parse action attached to the part with the format description to
create the flexible part of the parser, and to then insert it into the
placeholder Forward using the '<<' operator.
I touched up your parser from the gist that you posted, and have added
some sample code that will parse the test cube example. You can find it
here: https://gist.github.com/anonymous/4498de5ce91da8cf4292bad408076b97
I think you'll be able to take it from there. (I also applied some
stylistic changes - removing explicit .setResultsName calls in favor of
implicit callable notation, changing CaselessLiteral to CaselessKeyword, a
Group added here or there for better structure and results name assignment.
Feel free to keep any, all, or none of these changes.)
Please post a link to your finished project to this list, or the Pyparsing Facebook page.
Regards,
-- Paul McGuire