Parsing migration data in manual code

As an engine runs, it works through one Business Object at a time. Internally, the engine holds each Business Object as XML. That XML is not exposed to Manual Rules and Bags, and a Rule or Bag never works against it directly.

Instead, the engine generator produces a set of strongly-typed Parsers. A Parser is a friendly, navigable view of the Business Object the engine is currently working on. Through a Parser, a Rule or Bag reads the fields of the Business Object, walks to its children and its parent, and reaches the entities related to it, all in the names from the project.

This strong typing also protects a Rule or Bag as a project changes. The Parsers carry the names and types from the mapping, and they are regenerated along with the engine. If a field is renamed, or its type adjusted, code written against the old shape no longer compiles. The mismatch surfaces as a build error rather than as a quiet fault during a migration.

This article introduces the Parsers a Manual Rule or Bag can use, how to obtain one, and how to navigate it. For testing a Rule or Bag that uses a Parser, see Unit test of Manual Rules and Bags.

Reaching a Parser from a Rule or Bag

Every Manual Rule and Bag has a small set of Parsers available to it, ready to use. They arrive as properties on the generated base class that the Rule or Bag inherits from, so there is nothing to construct: you simply use them.

Which Parsers you have depends on the engine. A Rule or Bag in the Source Engine sees the Business Object on its way out of the source system; one in the Target Engine sees it on its way into the target. The table below lists what each engine offers:

Engine	Property	Reaches
Source Engine	SourceItem InterfaceItem	Source Parser Interface Parser (editable)
Target Engine	InterfaceItem TargetItem TargetItem.Enumerate Relationship	Interface Parser (read-only) Target Parser Target objects, reached directly Relationship Parser

The same set is available to every Rule type and to Bags in that engine.

The Source, Interface, and Target Parsers are all reached the same way. Call As() on the item, then pick the Business Object you want:

var account = InterfaceItem.As().Account.Parse();

As() gives you one property for every Business Object in the project. Selecting one returns a Parser provider. The provider is not yet the Parser: calling Parse() on it produces the Parser, and the Parser is the object you navigate.

The providers reach down the structure as well. Every child Business Object has its own provider, so you can go straight to a child Parser in a single expression:

var interest = InterfaceItem.As().Account.Interest.Parse();

The Relationship Parser and direct target-object access do not use As(). The sections below show how each is reached.

Parse, TryParse, and validity

A provider does not always have a Parser to give. The provider for Account.Interest only makes sense while the engine is processing an Interest that sits under an Account. Ask it for a Parser while the engine is busy with a Customer, and there is nothing for it to parse. A provider is valid when the Business Object it targets, and the whole path of Business Objects above it, match what the engine is processing at that moment.

Every provider offers two ways to produce a Parser, and they differ in how they treat an invalid provider:

IParser Parse(bool throwIfInvalid = true);
bool TryParse(out IParser parsed);

Parse() is the direct route. When the provider is valid it returns the Parser. When it is not, it throws an InvalidOperationException, unless you pass throwIfInvalid: false, in which case it returns null. TryParse() is the cautious route: it returns true and hands back the Parser when the provider is valid, and false with a null Parser when it is not.

Reach for Parse() in a Rule or Bag that only ever runs for a Business Object the provider fits. An exception there is a genuine signal that something is wrong. Reach for TryParse(), or Parse(false), when the same code may also run for a Business Object the provider does not match, and you would rather handle that case than fail.

Parent and child navigation

A Business Object rarely stands alone. It sits in a structure. An Account has Interest and Credit children. An Interest has InterestLine children, and so on down. Every Parser can move through that structure.

Children reaches the Business Objects below the current one. It has one member per kind of child. Each member is a sequence of Parsers, one for every child of that kind:

var account = InterfaceItem.As().Account.Parse();

foreach (var interest in account.Children.Interest)
{
  // interest is itself a Parser
}

Parent reaches the Business Object above. It is available on a child Business Object, and it returns the Parser of the parent:

var interest = InterfaceItem.As().Account.Interest.Parse();

var account = interest.Parent;

The children and parent you reach are Parsers themselves, of the same kind you started from. Navigate from an Interface Parser and you stay among Interface Parsers; a Target Parser behaves the same way. So you can keep going: a child has its own Children, and a parent has its own Parent.

The Source Parser navigates the same structure, with one difference. On a Source Parser, each Children member carries the copy layer. A child is reached through its copy, for example Children.Interest.Unnamed, or through a copy name where the child Business Object has been copied.

This is navigation within one Business Object structure. To reach a Business Object that is related but sits outside the structure, use the Relationship Parser instead.

Parsers in the Source Engine

A Manual Rule or Bag in the Source Engine has two Parsers to work with. The Source Parser shows the Business Object as it is read from the source system. The Interface Parser shows it as the engine will export it.

The Source Parser

The Source Parser is the earliest view of a Business Object. It shows the data as the Source Engine is reading it from the source system. The source representation is under construction; a Rule or Bag reads it in the state the engine has reached so far. It is read-only, and you reach it through the SourceItem property.

The Business Object

SourceItem.As() gives you one property per Business Object. Selecting one returns a Parser provider. The provider hands you the Parser through a named member.

A Business Object that has not been copied is reached through Unnamed:

var account = SourceItem.As().Account.Unnamed.Parse();

A Business Object can also be copied. A copy lets the same Business Object in the target interface be built from more than one extraction map. Each copy carries a name. The provider then offers one member per copy name in place of Unnamed.

The AddressComponent Business Object shows this. Below the physical address of a Customer, it is copied into Box, Building, CoName, Floor, Place, PostalCode, RoadName, and RoadNumber. Each copy has its own extraction map, and each is reached by its name:

var roadName = SourceItem.As().Customer.Unnamed
  .PhysicalAddress.Unnamed
  .AddressComponent.RoadName.Parse();

Once you have the Parser, it exposes Children for the child Business Objects, and an ExtractionMap for the source objects the Business Object is read from.

The extraction map

An extraction map describes how a Business Object is assembled from the source system. It is a tree of source objects: a root source object, the source objects joined below it, and so on down the tree.

Each source object offers three things. Fields exposes its typed fields. Children reaches the source objects below it. ParentSourceObject reaches the one above.

Start at the root source object and read its fields:

var sourceAccount = account.ExtractionMap.Account();

var bankId = sourceAccount.Fields.BankId;
var accountNumber = sourceAccount.Fields.AccountNumber;

Every source object accessor takes an optional includeDiscarded argument. By default, a source row that a Filter Rule has discarded is left out. Pass true to include discarded rows:

var sourceAccount = account.ExtractionMap.Account(includeDiscarded: true);

From there, walk down the tree with Children, and back up with ParentSourceObject:

// A source object joined below Account
var customer = sourceAccount.Children.Customer();
var customerNumber = customer.Fields.CustomerNumber;

// Children nest further down the tree
var clientAdditional = customer.Children.ClientAdditional();

// ParentSourceObject returns to the source object above
var owningAccount = customer.ParentSourceObject;

The names carry one more meaning. The same metadata entity can be added to an extraction map more than once. Each occurrence becomes a separate source object with its own name. Those names are aliases. They become the correlation ids of the source objects in the generated SQL Server stored procedure.

The Interface Parser

The Interface Parser shows a Business Object in its interface form. This is the shape in which the Source Engine exports it. The interface representation is under construction; a Rule or Bag reads it in the state the engine has reached so far. You reach it through the InterfaceItem property.

It exposes Fields, a typed property for each field of the Business Object. It also exposes Children, where each kind of child Business Object appears as a sequence of Parsers.

In the Source Engine, the Interface Parser is editable. You can modify a field, and the change is captured and shown in the Portal:

var account = InterfaceItem.As().Account.Parse();

// In the Source Engine, interface fields can be modified
account.Fields.BankUnit = account.Fields.BankUnit?.Trim();

Parsers in the Target Engine

A Manual Rule or Bag in the Target Engine has three Parsers to work with. The Interface Parser shows the imported interface representation. The Target Parser shows the Business Object on the target side. The Relationship Parser reaches the entities the Business Object is related to.

The Interface Parser

The Interface Parser shows a Business Object as it arrived from the source system. This is the interface representation that the Target Engine imports. It is complete: every field and every child is already there, exactly as the Source Engine produced it. You reach it through the InterfaceItem property. Here it is read-only.

What an Interface Parser exposes depends on where the Business Object sits in the structure:

• Fields provides a typed property for each field of the Business Object.
• Children provides each kind of child Business Object as a sequence of Parsers.
• Parent provides the Parser of the parent Business Object, on a child Business Object.
• IsImported, LineageId, PeerIndex, and ChildIndex provide metadata about the Business Object and its position among its siblings.

The example below is from the WorkshopDemo InterestLines Bag. It obtains the Parser for the Interest Business Object, then reads the InterestLine children and their fields:

var interest = InterfaceItem.As().Account.Interest.Parse();

var interestLines = interest.Children.InterestLine
  .Select(l => new
  {
    SeqNo = l.Fields.InterestLineSeqNo.GetValueOrDefault(),
    Limit = l.Fields.Limit.GetValueOrDefault()
  })
  .OrderBy(l => l.SeqNo);

Children.InterestLine is a sequence of Interface Parsers, one for each InterestLine child. Each one carries its own Fields. Here those fields are InterestLineSeqNo and Limit, both typed as nullable decimals.

The Target Parser

The Target Parser shows a Business Object on the target side, as the Target Engine builds it. The target representation is under construction: a Rule or Bag reads it in the state the engine has reached so far. You reach it through the TargetItem property. It is editable.

Like the other Parsers, it has Children and Parent. It adds one more member: Entities. Entities reaches the entities that the mapping produced for the Business Object. They are grouped under the alias of each Metadata in the Target Map.

The example below is from the WorkshopDemo CreateMissingInterests Exit Rule:

var account = TargetItem.As().Account.Parse();

// Entities reaches the entities; Tgt is the alias of a Metadata in the Target Map
var fields = account.Entities.Tgt.Account.First().Fields;

// Children walks to child Business Objects, each with its own Entities
var interests = account
  .Children.Interest.SelectMany(i => i.Entities.Tgt.Acc_Interest)
  .ToList();

Entities.Tgt groups the entities under the alias Tgt. Tgt is the alias given to a Metadata in the Target Map. A Target Map can hold more than one Metadata, each addressed by its own alias. Each entity exposes typed Fields. A discarded entity is not included. Children walks to the child Business Objects of the Account. Each child is itself a Target Parser, with its own Entities.

These target-system entities are editable. Assign to a field to modify the entity. Call Discard() on an entity to discard it. Both a field modification and a discard are captured and reported in the Portal.

Reaching target objects directly

The Target Parser reaches target objects by following the structure of the Business Object. Sometimes that is more work than you need. A Manual Rule or Bag can also reach target objects directly, through TargetItem.Enumerate.

TargetItem.Enumerate exposes one property per Metadata in the Target Map, under the same aliases used by Entities and Injectors. Tgt is one such alias. Each alias offers one method per target object type, returning every non-discarded instance of that type in the imported item:

// Every Acc_Interest target object in the imported item
var interests = TargetItem.Enumerate.Tgt.Acc_Interest();

Each method takes an optional TargetEntitySearchMode. It limits how far the search reaches:

TargetEntitySearchMode	Scope of the search
Global	The entire imported item. This is the default.
Item	The entities of the current item only.
ItemAndDescendants	The current item together with all of its descendant items.
Descendants	All descendant items of the current item.

// Acc_Interest target objects of the current item only
var interests = TargetItem.Enumerate.Tgt.Acc_Interest(TargetEntitySearchMode.Item);

The objects you get back are the same target objects you would reach through Entities on a Target Parser. They are editable in the same way.

Injection

Injection is how a Rule or Bag adds new target-system entities. Where Entities reads the entities the mapping produced, an Injectors property creates new ones.

Injectors is not on every Target Parser. It is generated for a Business Object only when its target entities have been made injectable in Studio. Where injection has not been expressed for an entity, that entity cannot be injected.

Each target-system entity under Injectors exposes an Inject method. The method takes one argument per field of the entity, passed as named arguments. The example below, again from the CreateMissingInterests Exit Rule, injects a new Acc_Interest entity into the Tgt target system:

account.Injectors.Tgt.Acc_Interest.Inject(
  BankId: fields.BankId.Value,
  AccountId: fields.AccountId.Value,
  InterestId: interestId,
  InterestRateType: row.InterestType,
  NumberOfInterestLines: 1,
  StartDate: fields.EffectiveDate.Value,
  EndDate: DateTime.MaxValue,
  CrudStatus: fields.CrudStatus.Value,
  CreatedBy: fields.CreatedBy.Value,
  CreatedTs: MappingRule.GetNewTs(n => true, "3", SystemConstant.TsDate),
  SystemId: fields.SystemId.Value
  );

Use named arguments

A named argument in C# names the parameter it binds to, written parameterName: value. A positional argument binds by the order in which the parameters were declared. The Inject calls above use named arguments throughout, and that is the form to use.

Inject is generated from the target metadata, and a change to the metadata can reorder its parameters. Named arguments stay correct across that change. Positional arguments do not: if two parameters of the same type swap places, the call still compiles but binds values to the wrong fields.

Injectors.Tgt mirrors Entities.Tgt. It is the same grouping by alias, used to write rather than to read. An injected entity is visible in the Portal.

The Relationship Parser

A Business Object is rarely alone. It is related to other Business Objects, and the Relationship Parser is how a Rule or Bag follows those relationships.

The Target Parser reaches the entities of the current Business Object. The Relationship Parser goes one step further. It reaches the Business Objects at the far end of a relationship. You reach it through the Relationship property, and it is read-only.

Relationship exposes one property per Business Object. Each relationship is reached through a Targets member:

// The Customer that is the primary owner of the Account
var primaryOwner = Relationship.Account.Targets.PrimaryOwner;

A relationship defined on a child Business Object is reached through that child:

// The Customer that receives the statements for the Account
var statementReceiver = Relationship.Account.Statement.Targets.StatementReceiver;

A Targets property does not hand back a Parser directly. It returns a relationship target, and that object is always present, whether or not the relationship is satisfied. Check its Exists property first.

When Exists is true, the related Business Object is available as a Parser in two forms. InterfaceItem.Parse() gives its Interface Parser, and TargetItem.Parse() gives its Target Parser:

if (primaryOwner.Exists)
{
  var ownerOnTarget = primaryOwner.TargetItem.Parse();
}

Calling Parse() when Exists is false throws, so the Exists check is not optional.

Testing Rules and Bags that use Parsers

A Rule or Bag that uses a Parser can still be unit tested. The Parser interfaces are public types, so a test can supply a Parser through a mocked context. See Unit test of Manual Rules and Bags for how.