validating NBNF

[gericson]

Does anyone have working examples of nBNF for the Grammar Editor?

[Eve]

I have a few that I can share. The following grammar is a completed version of a grammar example that is provided in the help.

Code: [Select]

// All required instructions
caseInsensitive();
delimiters(<DELIMITER>);
lex(<TOKENS>);
parse(<greeting>);

// Tokenization (lexing) rules
<DELIMITER> ::= <WHITESPACE>;
<TOKENS> ::= <WHITESPACE>+ | token( keywords() | <NAME> );
<WHITESPACE> ::= " " | "\r" | "\n" | "\t";
<NAME> ::= ("a".."z")+;

// Parsing rules
// attribute command ensures something is returned in the AST.
<greeting> ::= attribute("GREETING", "hello" | "hi" | ["good"] "morning");
Basically that will accept "Hello", "hi", "morning" or "good morning".

The next grammar is for a completely made up programming language. It's intended to be included in EA's code samples directory. Unfortunately it needs to be split across multiple posts because it's too long.

In order to use it to import code you would join all the parts together into a text file. Define a new programming language by adding at least one datatype. In the MDG technology wizard for a language you can select the programming language and select a grammar to go with it along with options including what file extensions to use. Check out the help because I believe it's covered there.


The first section defines the common instructions and tokenization rules. You'll find that this set will work as a basis for most programming languages.

Code: [Select]

// SSL - Simple Sample Language
//
// Words..

// Processing instructions
caseInsensitive();

delimiters(<DELIMITER>);

lex(<TOKENS>);

parse(<language>);

// Lexer rules
<TOKENS>                                                ::= <WHITESPACE> |
                                                                  <COMMENT> |
                                                                  token(<NUMBER>) |
                                                                  token("\"") token(<STRING_BODY>) token("\"") |
                                                                  token(keywords()) |
                                                                  token(<IDENTIFIER>);

<WHITESPACE>                                          ::= " " | "\r" | "\n" | "\t";

<COMMENT>                                                ::= <LINECOMMENT> |
                                                                  <BLOCKCOMMENT>;

<LINECOMMENT>                                          ::= "//" skip("\n") "\n" |
                                                                  "//" skipEof();

<BLOCKCOMMENT>                                          ::= "/*" skip("*/") "*/";

<NUMBER>                                                ::= <FLOAT_NUMBER> |
                                                                  <INT_NUMBER>;

<FLOAT_NUMBER>                                          ::= "0".."9"+ "." ["0".."9"+] [("e" | "E") ["+" | "-"] "0".."9"+] |
                                                                  "." "0".."9"+ [("e" | "E") ["+" | "-"] "0".."9"+] |
                                                                  "0".."9"+ ("e" | "E") ["+" | "-"] "0".."9"+ |
                                                                  "0".."9"+ [("e" | "E") ["+" | "-"] "0".."9"+];

<INT_NUMBER>                                          ::= <HEX_NUMBER> |
                                                                  <DEC_NUMBER>;


<HEX_NUMBER>                                          ::= "0x" ("0".."9" | "a".."f")+;

<DEC_NUMBER>                                          ::= "0".."9"+;

<STRING_BODY>                                          ::= skip("\"", "\\\"" | "\\\\");

<IDENTIFIER>                                          ::= ("a".."z" | "_") ("a".."z" | "0".."9" | "_")*;

<DELIMITER>                                                ::= " " | "\n" | "\r" | "\t" |
                                                                  "~" | "`"  | "!"  | "@"  |
                                                                  "#" | "$"  | "%"  | "^"  |
                                                                  "&" | "*"  | "("  | ")"  |
                                                                  "-" | "+"  | "="  | "{"  |
                                                                  "[" | "}"  | "]"  | ";"  |
                                                                  ":" | "\\" | "\"" | "\'" |
                                                                  "," | "<"  | ">"  | "/"  |
                                                                  "?" | "|"  | ".";

[Eve]

Next I'll include the code for parsing the language with the exception of expressions.

Code: [Select]

// Generic name rules
<simpleName>                                          ::= <IDENTIFIER>;

<qualifiedName>                                          ::= <IDENTIFIER> ("." <IDENTIFIER>)*;

// Generic type name rules
<typeName>                                                ::= <qualifiedName> [<templateSpec>];

<templateSpec>                                          ::= "<" <typeName> ">";


// Packages
<packageDeclaration>                              ::= node("PACKAGE", "package" <packageName> "{" <packageBody> "}");

<packageName>                                          ::= attribute("NAME", <simpleName>);

<packageBody>                                          ::= <packageScope>*;

<packageScope>                                          ::= <packageDeclaration> |
                                                                  <classDeclaration> |
                                                                  <interfaceDeclaration>;

// Classes
<classDeclaration>                                    ::= node("CLASS", node("DECLARATION", <classVisibility> "class" <className> [<classInherits>]) node("BODY", "{" <classBody> "}"));

<classVisibility>                                    ::= attribute("SCOPE", "public") |
                                                                  attribute("SCOPE", "protected") |
                                                                  attribute("SCOPE", "private") |
                                                                  attributeEx("SCOPE", "public");

<className>                                                ::= attribute("NAME", <simpleName>);

<classInherits>                                          ::= ":" attribute("PARENT", <qualifiedName>) ("," attribute("PARENT", <qualifiedName>))*;

<classBody>                                                ::= <classScope>*;

<classScope>                                          ::= <constructorDeclaration> |
                                                                  <destructorDeclaration> |
                                                                  <attributeDeclaration> |
                                                                  <operationDeclaration>;

// Interfaces
<interfaceDeclaration>                              ::= node("INTERFACE", <interfaceVisibility> node("DECLARATION", "interface" <interfaceName> [<interfaceInherits>]) node("BODY", "{" <interfaceBody> "}"));

<interfaceVisibility>                              ::= attribute("SCOPE", "public") |
                                                                  attribute("SCOPE", "protected") |
                                                                  attribute("SCOPE", "private") |
                                                                  attributeEx("SCOPE", "public");

<interfaceName>                                          ::= <simpleName>;

<interfaceInherits>                                    ::= ":" <qualifiedName> ("," <qualifiedName>)*;

<interfaceBody>                                          ::= <interfaceScope>*;

<interfaceScope>                                    ::= <operationDeclaration>;

// Constructors
<constructorDeclaration>                        ::= node("METHOD", attributeEx("SCOPE", "public") attribute("NAME", <simpleName>) "(" [<parameterList>] ")" [<constructorBody>] ";");

<constructorBody>                                    ::= "{" <statement>* "}";

// Destructors
<destructorDeclaration>                              ::= node("METHOD", attributeEx("SCOPE", "public") attribute("NAME", "~" <simpleName>) "(" ")" [<destructorBody>] ";");

<destructorBody>                                    ::= "{" <statement>* "}";

// Attributes
<attributeDeclaration>                              ::= node("FIELD", [<attributeVisibility>] node("TYPE", <attributeType>) node("DECLARATOR", <attributeName>) [<attributeDefault>] ";");

<attributeVisibility>                              ::= attribute("SCOPE", "public") |
                                                                  attribute("SCOPE", "protected") |
                                                                  attribute("SCOPE", "private") |
                                                                  attributeEx("SCOPE", "public");

<attributeType>                                          ::= attribute("TYPE", <typeName>);

<attributeName>                                          ::= attribute("NAME", <simpleName>);

<attributeDefault>                                    ::= node("DEFAULT", "=" attribute("VALUE", <constantExpression>));

// Operations
<operationDeclaration>                              ::= node("METHOD", node("DECLARATION", <operationVisibility> <operationType> <operationName> <operationParameters>) [<operationBody>] ";");

<operationVisibility>                              ::= attribute("SCOPE", "public") |
                                                                  attribute("SCOPE", "protected") |
                                                                  attribute("SCOPE", "private") |
                                                                  attributeEx("SCOPE", "public");

<operationType>                                          ::= attribute("TYPE", <typeName>);

<operationName>                                          ::= attribute("NAME", <simpleName>);

<operationParameters>                              ::= "(" [<parameterList>] ")";

<parameterList>                                          ::= <parameter> ("," <parameter>)*;

<parameter>                                                ::= node("PARAMETER", <parameterType> <parameterName>);

<parameterType>                                          ::= attribute("TYPE", <typeName>);

<parameterName>                                          ::= attribute("NAME", <simpleName>);

<operationBody>                                          ::= "{" <statement>* "}";

// Statements
<statement>                                                ::= <variableDeclarationStatement> |
                                                                  <assignmentExpression>  ";"|
                                                                  <methodAccess> ";";

<variableDeclarationStatement>                  ::= <variableType> <variableName> [<variableDefault>] ";";

<variableType>                                          ::= <qualifiedName>;

<variableName>                                          ::= <simpleName>;

<variableDefault>                                    ::= "=" <constantExpression>;

[Eve]

Finally, parsing expressions.

Again, much of this will be common to most programming languages.

Code: [Select]

// Expressions
<assignmentExpression>                              ::= preProcess(<primaryExpression>,
                                                                        <?> "=" <constantExpression> |
                                                                        <?> "+=" <constantExpression> |
                                                                        <?> "-=" <constantExpression>
                                                                  );

<constantExpression>                              ::= <conditionalExpression>;

<conditionalExpression>                              ::= preProcess(<logicalOrExpression>,
                                                                        <?> "?" <conditionalExpression> ":" <conditionalExpression> |
                                                                        <?>
                                                                  );

<logicalOrExpression>                              ::= preProcess(<logicalAndExpression>,
                                                                        <?> "||" <logicalOrExpression> |
                                                                        <?>
                                                                  );

<logicalAndExpression>                              ::= preProcess(<inclusiveOrExpression>,
                                                                        <?> "&&" <logicalAndExpression> |
                                                                        <?>
                                                                  );

<inclusiveOrExpression>                              ::= preProcess(<exclusiveOrExpression>,
                                                                        <?> "|" <inclusiveOrExpression> |
                                                                        <?>
                                                                  );

<exclusiveOrExpression>                              ::= preProcess(<andExpression>,
                                                                        <?> "^" <exclusiveOrExpression> |
                                                                        <?>
                                                                  );

<andExpression>                                          ::= preProcess(<equalityExpression>,
                                                                        <?> "&" <andExpression> |
                                                                        <?>
                                                                  );

<equalityExpression>                              ::= preProcess(<relationalExpression>,
                                                                        <?> "==" <equalityExpression> |
                                                                        <?> "!=" <equalityExpression> |
                                                                        <?>
                                                                  );

<relationalExpression>                              ::= preProcess(<shiftExpression>,
                                                                        <?> "<" <relationalExpression> |
                                                                        <?> ">" <relationalExpression> |
                                                                        <?> "<=" <relationalExpression> |
                                                                        <?> ">=" <relationalExpression> |
                                                                        <?>
                                                                  );

<shiftExpression>                                    ::= preProcess(<additiveExpression>,
                                                                        <?> "<<" <shiftExpression> |
                                                                        <?> ">>" <shiftExpression> |
                                                                        <?>
                                                                  );

<additiveExpression>                              ::= preProcess(<multiplicativeExpression>,
                                                                        <?> "+" <additiveExpression> |
                                                                        <?> "-" <additiveExpression> |
                                                                        <?>
                                                                  );

<multiplicativeExpression>                        ::= preProcess(<primaryExpression>,
                                                                        <?> "*" <multiplicativeExpression> |
                                                                        <?> "/" <multiplicativeExpression> |
                                                                        <?> "%" <multiplicativeExpression> |
                                                                        <?>
                                                                  );

<primaryExpression>                                    ::= <literal> |
                                                                  <methodAccess> |
                                                                  <variableAccess> |
                                                                  "(" <constantExpression> ")";

<methodAccess>                                          ::= <qualifiedName> "(" [<argumentList>] ")";

<argumentList>                                          ::= <argument> ("," <argument>)*;

<argument>                                                ::= <constantExpression>;

<variableAccess>                                    ::= <qualifiedName>;

// Literals
<literal>                                                ::= <numericLiteral> |
                                                                  <stringLiteral>;

<numericLiteral>                                    ::= <NUMBER>;

<stringLiteral>                                          ::= "\"" <STRING_BODY> "\"";

[Eve]

For a final code snippet, here's an example of the (made up) language that's parsing.

Code: [Select]

// Test.SSL

package Sparx {
      package EA {
            public interface Foo
            {
                  int bar();
            }
            
            public class Element : Sparx.PersistedObject, Sparx.DrawnObject {
                  Element()
                  {
                        options = 0.01;
                        connectionString = "Hello, world!";
                  };

                  ~Element()
                  {
                  };
                  
                  private String connectionString;
                  private Long options;
                  
                  public Boolean Connect(String<utf8> ConnectionString, Long Options)
                  {
                        this.that.theOther.Connect("JFDI", ConnectionString.ToNonString(), ConvertInSomeManner(Options));
                  };
                  
                  public Boolean Disconnect()
                  {
                  };
                  
                  public Sparx.EA.ElementList RelatedElements = null;
            }
      }
}


[g.makulik]

THX Simon that's great! I'll be going to play with this stuff   ...

I know I'm greedy  , but can you give some thoughts about this question now?

Will it be possible to 'override' the grammar of EA's supported standard languages, by means of changing the grammar definitions project wise? Similar as possible with the code generation templates.

This would be pretty useful to support particular language features or dialects, that aren't supported natively by EA reverse engineering/code synchronization.

I couldn't see an obvious way from my 1st (rough) research on the docs and look at the grammar editor.

Best regards,
Günther

PS.: I guess EA doesn't actually use this feature internally to parse the already provided programming languages. But it would be nice to have a way to intercept/extend these.

[qwerty]

A *BNF (context free grammar) only describes the valid syntax of a language. It does not tell anything about semantics. So, Günther, I guess there wont come much more around that way. (Of course an open parser would be some nice stuff for academic research, but YACC and those guys are a bit beastly.)

q.

[g.makulik]

It does not tell anything about semantics. So, Günther, I guess there wont come much more around that way.

My point isn't to extract more 'high level' semantics from reverse engineering, but keeping extensions of code generation (that fill gaps for language features that aren't supported directly by EA) in sync with the code. I'm talking about C/C++ language actually, and it would be nice being able to extend the parser for these languages, instead of creating your own parser rules for a 'new' version of C/C++. This needs to have a different language name then as far I understood from the docs so far. But may be I completely misunderstood how it's intended to work, more clarification is appreciated.

Best regards,
Günther

[Eve]

Günther,

At this stage we aren't supporting that use case. We do use something very similar internally, in fact we can often provide an MDG technology that patches a grammar for a supported language. I understand why you want to extend the C/C++ grammars. What in particular are you trying to handle?

Qwerty,

A pure BNF describes valid syntax only. Sparx Systems uses a variant with extensions that allow an Abstract Syntax Tree (AST) to be created. To demonstrate I'll show the part of the grammar from above for attribute parsing. Note the node and attribute calls. This generates the bit that EA uses to generate a UML model and is the equivalent of inserting executable code in YACC.

Code: [Select]

<attributeDeclaration> ::= node("FIELD", [<attributeVisibility>] node("TYPE", <attributeType>) node("DECLARATOR", <attributeName>) [<attributeDefault>] ";");

<attributeVisibility> ::= attribute("SCOPE", "public") |                                                        attribute("SCOPE", "protected") |
                          attribute("SCOPE", "private") |
                          attributeEx("SCOPE", "public");

<attributeType> ::= attribute("TYPE", <typeName>);
<attributeName> ::= attribute("NAME", <simpleName>);
<attributeDefault> ::= node("DEFAULT", "=" attribute("VALUE", <constantExpression>));

[g.makulik]

What in particular are you trying to handle?

This case for instance: http://www.sparxsystems.com/cgi-bin/yabb/YaBB.cgi?num=1353618338.
I have a solution, but it's clumsy, doesn't produce well readable code and can't handle all cases so far.

Best regards,
Günther

[qwerty]

Sparx Systems uses a variant with extensions that allow an Abstract Syntax Tree (AST) to be created.

So that means your variant is actually context sensitive. What does the first N in NBNF stand for?

q.

[g.makulik]

So that means your variant is actually context sensitive. What does the first N in NBNF stand for?

It is, since you have access to the AST when certain actions should be done with certain nodes in the AST (same like you do with YACC to get some meaningful output).

Can't tell about the 'N', didn't seen this mentioned in EA docs either:

Grammars define how a text is to be broken up into a structure, which is exactly what is needed when you are converting code into a UML representation. At the simplest level, a grammar is just instructions for breaking up an input to form a structure. Enterprise Architect uses a variation of Backus–Naur Form (BNF) to express a grammar in a way that allows it to convert the text to a UML representation. What the grammar from Enterprise Architect offers over a pure BNF is the addition of processing instructions, which allow structured information to be returned from the parsed results in the form of an Abstract Syntax Tree (AST). At the completion of the AST, Enterprise Architect will process it to produce a UML model.

[Eve]

Looks like that's nearly entirely code gen templates, with just a couple of differences to reversing to ensure that code is imported how you want to export it.

Unfortunately I think that C code requires some post-processing by EA to import and I'm not sure that updating the grammar would be enough to do what you want to do.

[Eve]

I don't have a definitive meaning for the 'N'. I can say that it's not anything more meaningful than 'new', which is what I have thought of it as.

When parsing you don't have access to the AST, and strictly speaking I believe it's still a context free grammar.

[g.makulik]

Looks like that's nearly entirely code gen templates, with just a couple of differences to reversing to ensure that code is imported how you want to export it.

Unfortunately I think that C code requires some post-processing by EA to import and I'm not sure that updating the grammar would be enough to do what you want to do.

I've been thinking this over. The kind of problem described in the mentioned thread boils down to the fact that there's no distinction of an UML attributes implementation in header and source file. If this could be handled by an extension of the parsing rules and how these apply in the context of either header or source file, is questionable yes.

May be the more promising way is to post process the synchronized model using AddIn code or scripting. Am I on the right track with MDG Events, MDG_PostMerge in particular?
I'm also not so enlightened about the whole MDG_Connect mechanism, any samples would be appreciated.

Best regards,
Günther