This section documents IRIS Ontology and Knowledge Base access. A "Knowledge Base" is essentially an ontolgoy database in IRIS.
RQ-CORE-INTEGRATION implies a standard data representation. The standard data representation in IRIS is OWL. The OWL spec that IRIS uses is a subset of the CLIB upper ontology. As IRIS is targeted towards the office work environment domain, the core-plus-office subset of CLIB is used.
IRIS use the following OWL specification files:
The core-plus-office subset of CLIB. This is the "upper ontology" that IRIS uses. Whenever data is used by IRIS, it is converted into this upper ontology first.
Contains OWL clases and properties used in IRIS that supplement the CLIB ontology.
IRIS uses an ontology-based messaging system known as PubSub. This file contains the PubSub task ontology.
Contains RDF data used to display UI in IRIS data forms.
Ontology for IRIS harvesting framework and data source specification.
Ontology used for data normalization using the IRIS Identity Server.
Ontology path definitions which facilitate ontology access.
Not currently used. This ontology is used for the collab plugin which is deprecated.
An ontology which fomally specifies form definitions in OWL format. This is a prototype and is not currently used. It is a future roadmap for replacing irisAuxillary.rdf.
Tracks ontolgoy changes using SOUP (Simple Ontology Update Program)
Contains task parameters. This file is currently just copied routinely from task parameter generation.
Protege is capable of importing the OWL files directly and automatically resolves imports. The following instructions import any of the OWL files for viewing into Protege, and have been tested with Protege version 3.2.1:
After launching Protege, click "New Project..."
Make sure "Create from existing sources" is checked at the top and select "OWL/RDF files". Click "Next"
Browse for the OWL file you want to load. For example, iris/data/harvestowl. Click "Finish".
If Protege assk to resolve references to other ontologies, set a relative directory of "."
Protege also comes with a viewer namedOWLViz. In order to use OWLViz, you need to make sure that GraphViz is inatlled (Note: GraphViz comes with the Eclipse contraint reasoner, so most people already have it). To start OWLViz, go to Project->Configure and make sure "OWLVizTab" is checked. Then you can go into OWLViz and display a class.
In the iris/data/inProgress directory are working in-progress ontology files for CLib and IRIS. These files are used when importing new ontology information into IRIS. For example, when a new version of CLib is released, it will typically be copied to the inProgress directory before being transferred to the iris/data directory. Similarly, pending changes to irisOntology.owl are made here also. In fact, as a general rule anytime these ontology files need to be changed they must be changed only in the "inProgress" directory. After that there are ant tasks which will properly transfer the files to the iris/data directory and also do some processing on those files in order to keep track of ontology version information in IRIS. This transferring process using ant tasks is detailed in the SOUP documentation. Note that the ontology files in the inProgress directory can be viewed and edited using the same instructions above.
Ontology usage documentation provides the following:
Documents what ontology different IRIS plugins are using. Other IRIS developers can view the documentation to find out what ontology is used, and why.
The CLIB upper ontology is large, and ontology usage narrows down the subset of ontology being used.
IRIS generates POJOs which are programming interfaces. The POJOs are generated from ontology usage documentation.
A sample template file is located at "iris/data/docs/template.owl". Copy this template file and name the copied file according to the module name (ex: "Browser.owl"). In order to view the ontology documentation Mozilla Firefox 1.0 or higher should be installed. This template file is self documenting. Vew it in both html (by opening it in Firefox) and also as text in a text editor (or XML Editor).
Note that it is not necessary to document the author for all nodes. The only nodes the author is required to be documented for are:
The Documentation instance for a ModuleDoc.
The Documentation instance for a ScenarioDoc
If the author is missing for any other documentation, it is automatically assumed the author of the parent node is the same author.
Note that it is not required to put the "rdf:ID" (aka "name") attribute on every xml element. The XML elements which do require the "rdf:ID" attribute are:
Module: The id should not contain spaces and should be brief (ex: MailApp, Browser)
Scenario: The id should be "[MODULE_NAME]_[SCENARIO_NAME]". For example, "MailApp_HarvestMail"
KnownIssue
TestProcedure
Note that it is possible to define the html markup inside a "doc:hasMarkedUpContent" property using a CData section
Ontology usage is ducmented per scenario. When documenting ontology usage, documentation for the class that is used is provided, and documentation for all properties used on that class is provided. For example, here is the documentation for MailApp's usage of the class http://calo.sri.com/core-plus-office#EmailThread and what slots are used on that class:
378 <doc:hasOntologyDocumentation> 379 <doc:OntologyClassDoc> 380 <doc:hasOntologyClass rdf:resource="http://calo.sri.com/core-plus-office#EmailThread"/> 381 <doc:hasMarkedUpContent>Reifies a thread of email messages</doc:hasMarkedUpContent> 382 <doc:hasRelatedPropertyDoc> 383 <doc:OntologyPropertyDoc> 384 <iris:hasOntologyProperty 385 rdf:resource="http://calo.sri.com/core-plus-office#first-element"/> 386 <iris:hasRestrictiveClass 387 rdf:resource="http://calo.sri.com/core-plus-office#EmailMessage"/> 388 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 389 <doc:hasMarkedUpContent>The first message in the email thread.</doc:hasMarkedUpContent> 390 </doc:OntologyPropertyDoc> 391 </doc:hasRelatedPropertyDoc> 392 </doc:OntologyClassDoc> 393 </doc:hasOntologyDocumentation>
Note that this element should be placed as a child of the "doc:Scenario" element.
Suppose that one Scenario uses another Scenario? For example, a File_Harvest scenario might be used by other scenarios which need to harvest a file, but also populate other ontology data besides just the file harvesting. In this case, it is possible to add a "doc:hasScenarioUsage" element as a child of the "doc:Scenario". For example:
785 <doc:hasScenarioUsage> 786 <doc:ScenarioUsage> 787 <doc:hasDocumentation> 788 <doc:Documentation> 789 <doc:hasMarkedUpContent>If present, the common metadata for all OLE 2 Compound Documents 790 is extracted.</doc:hasMarkedUpContent> 791 </doc:Documentation> 792 </doc:hasDocumentation> 793 <doc:hasScenario 794 rdf:resource="http://iris.sri.com/irisOntology#OnlineSemex_HarvestOfficeDocument"/> 795 </doc:ScenarioUsage> 796 </doc:hasScenarioUsage>
Note that ScenarioUsage must contain a Documentation instance which documents how/when the scenario is used. When viewing the ontology documentation, the scenario usage will be displayed first in the Scenario's outputted html docuemntation.
See Binding Paths for information about a binding path. Note that an OntologyPropertyDoc may refer to the URI for a binding path which starts from the class for the ontology property documentation.
A Scenario can have zero or more instrumented events documented. An instrumented event is an event that the module sends through the IRIS instrumentation API. The following is an example of event documentation:
1063 <doc:hasEventProduction> 1064 <doc:EventClassDoc rdf:ID="MailApp_OpenComposeEmailWindow"> 1065 <doc:hasOntologyClass 1066 rdf:resource="http://calo.sri.com/tasks#OpenComposeEmailWindow"/> 1067 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 1068 <doc:hasMarkedUpContent>Sent when the user opens a new compose email window. Note that interpreting composeType 1069 for automation is not done yet</doc:hasMarkedUpContent> 1070 <doc:producesInstrumentation rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:producesInstrumentation> 1071 <doc:consumesAutomation rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:consumesAutomation> 1072 <doc:hasEventPropertyDoc> 1073 <doc:EventPropertyDoc> 1074 <iris:hasOntologyProperty rdf:resource="http://calo.sri.com/tasks#hasPerson"/> 1075 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 1076 <doc:hasMarkedUpContent>The person to compose an email to</doc:hasMarkedUpContent> 1077 <doc:isInputProperty rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:isInputProperty> 1078 <doc:isOutputProperty rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">false</doc:isOutputProperty> 1079 <doc:isOptionalOutput rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">false</doc:isOptionalOutput> 1080 <doc:isOptionalInput rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:isOptionalInput> 1081 </doc:EventPropertyDoc> 1082 </doc:hasEventPropertyDoc> 1083 <doc:hasEventPropertyDoc> 1084 <doc:EventPropertyDoc> 1085 <iris:hasOntologyProperty rdf:resource="http://calo.sri.com/tasks#hasEmail"/> 1086 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 1087 <doc:hasMarkedUpContent>A template email to fill out the recipients, attachments, subject and body</doc:hasMarkedUpContent> 1088 <doc:isInputProperty rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:isInputProperty> 1089 <doc:isOptionalInput rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:isOptionalInput> 1090 </doc:EventPropertyDoc> 1091 </doc:hasEventPropertyDoc> 1092 <doc:hasEventPropertyDoc> 1093 <doc:EventPropertyDoc> 1094 <iris:hasOntologyProperty 1095 rdf:resource="http://calo.sri.com/tasks#hasComposeType"/> 1096 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 1097 <doc:isOptionalInput rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:isOptionalInput> 1098 <doc:isInputProperty rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:isInputProperty> 1099 <doc:hasMarkedUpContent> 1100 <![CDATA[<p> 1101 Indicates the type of compose window. Type can be: 1102 <ul> 1103 <li>New = 0;</li> 1104 <li>NewsPost = 5;</li> 1105 <li>Draft = 9;</li> 1106 <li>Template = 10;</li> 1107 <li>MailToUrl = 11;</li> 1108 </ul> 1109 </p> 1110 ]]> 1111 </doc:hasMarkedUpContent> 1112 </doc:EventPropertyDoc> 1113 </doc:hasEventPropertyDoc> 1114 <doc:hasEventPropertyDoc> 1115 <doc:EventPropertyDoc> 1116 <iris:hasOntologyProperty 1117 rdf:resource="http://calo.sri.com/tasks#emailAttachmentFilePathIs"/> 1118 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 1119 <doc:hasMarkedUpContent>Convenience parameter that specifies the paths to all attachments. 1120 The paths can also be traversed through the attachment records to get the ComputerFile instance. 1121 Note that this property is output for instrumentation, but not set wehn doing automation 1122 (but it can be input for automation as a convenience)</doc:hasMarkedUpContent> 1123 <doc:isOptionalInput rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:isOptionalInput> 1124 <doc:isInputProperty rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</doc:isInputProperty> 1125 </doc:EventPropertyDoc> 1126 </doc:hasEventPropertyDoc> 1127 <doc:testProcedureIs> 1128 <doc:TestProcedure> 1129 <doc:hasName>OpenComposeEmailWindow Setup</doc:hasName> 1130 <doc:hasDocumentation> 1131 <doc:Documentation> 1132 <doc:hasMarkedUpContent> 1133 <![CDATA[ 1134 To test that OpenComposeEmailWindow works properly, the best way to test is to load the following beanshell script into IRIS: 1135 <a target="newin" href="../../plugins/thunderbird/doc/beanShell/EmailInstrumentationListener.txt">EmailInstrumentationListener.txt</a>. 1136 Only load this beanShell script once into IRIS (ie, do not try to load it twice or there will be too much debug output). 1137 <p> 1138 Before testing, restart Thunderbird. This is necessary to ensure a clean test of recycle listeners. 1139 </p> 1140 ]]> 1141 </doc:hasMarkedUpContent> 1142 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 1143 </doc:Documentation> 1144 </doc:hasDocumentation> 1145 </doc:TestProcedure> 1146 </doc:testProcedureIs> 1147 <doc:knownIssueIs> 1148 <doc:KnownIssue rdf:ID="MailApp_OpenComposeEmailWindow_KnownIssues_ComposeType"> 1149 <doc:hasName>OpenComposeEmailWindow automation doesn't interpret compose type</doc:hasName> 1150 <doc:issueTrackerLinkIs>https://jira.esd.sri.com/browse/IRI-693</doc:issueTrackerLinkIs> 1151 <doc:hasDocumentation> 1152 <doc:Documentation> 1153 <doc:hasMarkedUpContent> 1154 <![CDATA[ 1155 OpenComposeEmailWindow automation does not interpret the compose type. It opens a compose window as type 0 currently, and so 1156 other types like mailto, draft, template, etc. are not currently supported. 1157 ]]> 1158 </doc:hasMarkedUpContent> 1159 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 1160 </doc:Documentation> 1161 </doc:hasDocumentation> 1162 </doc:KnownIssue> 1163 </doc:knownIssueIs> 1164 <doc:resolvedIssueIs> 1165 <doc:KnownIssue rdf:ID="MailApp_OpenComposeEmailWindow_KnownIssues_Automation_Duplicate"> 1166 <doc:hasName>Thunderbird automation results in duplicate instrumentation</doc:hasName> 1167 <doc:issueTrackerLinkIs>https://jira.esd.sri.com/browse/IRI-726</doc:issueTrackerLinkIs> 1168 <doc:hasDocumentation> 1169 <doc:Documentation> 1170 <doc:hasMarkedUpContent> 1171 <![CDATA[ 1172 many thunderbird events result in duplicated instrumentation that indicates the user performed the event. For example, OpenComposeEmailWindow and OpenComposeResponseWindow 1173 ]]> 1174 </doc:hasMarkedUpContent> 1175 <doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName> 1176 </doc:Documentation> 1177 </doc:hasDocumentation> 1178 </doc:KnownIssue> 1179 </doc:resolvedIssueIs> 1180 </doc:EventClassDoc> 1181 </doc:hasEventProduction>
The "hasEventProduction" may be declared after the "hasOntologyDocumentation." The "doc:EventClassDoc" element specifies documentation, the author, "doc:EventPropertyDoc" children, and specify (note: default is false for all the true/false elements):
(true/false) whether or not instrumentation is sent out for the event
(true/false) whether or not automation is accepted and executed for the event
(true/false) whether or not this event is sent out for automation to another CALO component.
(true/false) if an event is being documented only because it is a common base class for other events, or POJOs need to be generated but the event instrumentation or automation is not yet implemented, set isAbstract to true.
Must be set that is unique for all ontology usage documentation.
An Event in IRIS is comprised of one or more "doc:EventPropertyDoc" elements. Each of these specifies the ontology property URI, documentation, author and (default is false if any of these are not specified):
(true/false) indicates whether or not the property is input in an automation event (if this event is consumed for automation or automation is sent out). Note that if a property is set on an instrumentation event and that is a property that would normally be input for automation, even though automation may not be implemented yet the property should be marked as an input property.
(true/false) whether or not the input property is optional
(true/false) whether or not the property is specified in automation output
(true/false) whether or not the output property is optional
In addition to property documentation, one or more test cases should be documented as "doc:testProcedureIs/doc:TestProcedure" elements. Each "doc:TestProcedure" must contain:
The documentation for the test procedure. The documentation should refer to any resources necessary to perform the test. For example, IRIS BeanShell scripts. The references should also be relative links if possible (use "../.." to get to the IRIS root).
Any known issues should also be documented as "doc:knownIssueIs/doc:KnownIssue" elements which contains:
A summary of the known issue
Unique for all ontology usage documentation.
A link to the issue tracker issue for the known issue
When a known issue is resolved, do not remove it. Instead change the "doc:knownIssueIs" property to be "doc:doc:resolvedIssueIs". Finally, note that a Scenario can also document known and resolved issues (as children of the Scenario element, doc:knownIssuesIs and doc:resolvedIssueIs).
There are several ways to test the documentation. The easiest way to test documentation is to open up the OWL file in Firefox which will present modue docuemntation for only that module. When the documentation is complete, add it to the master index and make sure it displays correctly.
The other way to test documentation is by validating the ontology. The IRIS Build does this automatically, or running "ant validate-ontology" from iris/src will also perform validation for all OWL files under iris/data (recursively).
Once you are finished writing the instance file, have tested it and are satisifed, edit the file iris/data/docs/allDocumentation.xml and add a "doc:includeOWL" element which refers to the OWL file name you have created. Also remember to add the OWL file to CVS.
Use this link to view ontology usage online.
Note that viewing ontology usage documentation is only tested with Firefox.See Pizza-doc.owl for documentation on how to import a custom ontology and write ontology usage documentation. Note that this OWL file must be viewed in Firefox.
POJOS (Plain Ordinary Java Objects, or Programmable Ontology Java Objects) refer to Java and C# interfaces which expose a programming API for ontology objects. This document describes how to generate POJOs. How to use POJOs is documented later.
A Task Parameter instance is an instanceof the class http://calo.sri.com/tasks#Parameter. Task Parameter instances are used by the Task learning Research systems in CALO, and they formally describe the parameters that gets set in PubSub Automation commands. This document describes how task parameters are generated.
The IRIS Build generates POJOs automatically. When writing ontology usage documentation, POJOs can be re-generated automatically by runnig the ant task ant generate-pojos from iris/src.
Internally POJOs are generated by running the generate-pojos-from-docs ant task in run-sdk.xml. For documentation on the task parameters, see the Javadoc for PojoGeneratorTask. This task outputs Java and C# interfaces to the iris/dest/pojo directory. The Java doc for POJOs can be found here. Note that no implementations are written for POJOs. IRIS Java and C# libraries use AOP to auto-generate POJO implementation classes on the fly. This is done for a number of reasons:
If an individual has more than one rdf type, it will implement all the POJO interfaces.
POJOs are used for remoting using the proxy plugin. Thr remoting layer is transparent and encapsulated from the client.
There are a lof of POJO classes and eliminating implementation classes reduces the overall footprint of the output libraries.
POJO interfaces are annotated so that the AOP invocation handlers in Java and .NET can properly delegate the invocations. For example, consider the IEMailMessagePojo class:
25 @SOMInterface(rdfClasses={IEmailMessagePojo.URI},transactional=false)
26 public interface IEmailMessagePojoThe SOMInterface attribute describes the base RDF types and also whether or not the POJO is transactional.
Each POJO interface has the URI of the class (which is the first rdf class listed). It also has constants for every property and 5 methods generated as follows:
31 static final String URI = "http://calo.sri.com/core-plus-office#EmailMessage"; 32 33 static final String FORWARDED_EMAIL_IS = "http://calo.sri.com/core-plus-office#forwardedEmailIs"; 34 35 @SOMProperty(rdfProperty=FORWARDED_EMAIL_IS,methodType=SOMProperty.MethodType.GET) 36 com.sri.iris.store.pojo.IEmailMessagePojo getForwardedEmailIs(); 37 38 @SOMProperty(rdfProperty=FORWARDED_EMAIL_IS,methodType=SOMProperty.MethodType.GET_ALL) 39 com.sri.iris.store.pojo.IEmailMessagePojo[] getAllForwardedEmailIs(); 40 41 @SOMProperty(rdfProperty=FORWARDED_EMAIL_IS,methodType=SOMProperty.MethodType.SET) 42 void setForwardedEmailIs(com.sri.iris.store.pojo.IEmailMessagePojo value); 43 44 @SOMProperty(rdfProperty=FORWARDED_EMAIL_IS,methodType=SOMProperty.MethodType.ADD) 45 void addForwardedEmailIs(com.sri.iris.store.pojo.IEmailMessagePojo value); 46 47 @SOMProperty(rdfProperty=FORWARDED_EMAIL_IS,methodType=SOMProperty.MethodType.DELETE) 48 void deleteForwardedEmailIs(com.sri.iris.store.pojo.IEmailMessagePojo value);
The SOMProperty annotation documents the proeprty URI and the type of method (GET, GET_ALL, SET, ADD or DELETE).
For C# the same structure is output, expect that C# does not allow constants in interface definitions. For this reason constants are defined in another class with the same name as the POJO class and "Constants" appended (ex: IEmailMessagePojoConstants).
At build time PojoGeneratorTask outputs all POJO mappings to a file located at iris/dest/pojo/src/java/com/sri/iris/store/pojo/generated_pojo_mappings.txt. .This is a serialized Java properties file that maps class URIs to POJO interfaces. For example:
http\://www.radarnetworks.com/core\#User=pojo.www.radarnetworks.com.core.IUserPojo http\://iris.sri.com/harvest\#FileSystemFolder=pojo.iris.sri.com.harvest.IFileSystemFolderPojo http\://calo.sri.com/core-plus-office\#CalendarWCAPAccount=com.sri.iris.store.pojo.ICalendarWCAPAccountPojo http\://iris.sri.com/harvest\#ThunderbirdEmailHeader=pojo.iris.sri.com.harvest.IThunderbirdEmailHeaderPojo http\://iris.sri.com/irisOntology\#IntParameter=pojo.iris.sri.com.irisOntology.IIntParameterPojo http\://iris.sri.com/irisOntology\#Cluster=pojo.iris.sri.com.irisOntology.IClusterPojo ...
At runtime, com.sri.iris.store.pojo.PojoMappingLoader loads this file and adds POJO mappings using the method addClassMapping on com.radarnetworks.kb.KnowledgeBase. At runtime POJOs are created and cached by com.sri.iris.jena.SemanticObjectFactory. The work of creating AOP implementations for POJOs is done by com.sri.som.pojo.EnhancedPojoFactory.
For an example of creating POJOs programatically in-memory and controlling the implementation, see the source code forcom.sri.som.pojo.MemoryPojoFactoryImpl.
Task Parameters are generated by the IRIS build automatically. The IRIS build does this by running the generate-task-params-from-docs ant task in run-sdk.xml. For documentation on the task parameters, see the Javadoc for TaskParamsGeneratorTask.
Task parameters are written to the file iris/dest/owl/taskParams.owl. The task also queries all Parameter instances and writes the results as XML to the file iris/dest/owl/taskParams.xml. Note that when task parameters are geenrated, they are only generated for classes that do not already have task parameters. The use case is that task parameters are generated for missing parameters, and then routinely copied to the persistant task parameters file located at iris/data/taskParams.owl. To view and test the persistant task parameters in Protege:
Open iris/data/taskParams.owl in Protege using version 3.2.1 or later by creating a new project of type "OWL/RDF Files" and checking the "Create from existing sources" checkbox. Then specify taskParams.owl as the existing file to be imported.
Click finish. Protege will complain that it cannot find the tasks ontology. When this happens, add a Repository with a Relative folder that points to ".".
Once Protege has loaded, choose the "OWL->Open SPARQL Query Panel..." menu item.
The following SPARQL query will query all the information for task parameters:
PREFIX tasks: <http://calo.sri.com/tasks#>
SELECT DISTINCT ?taskParam ?objType ?taskParamName ?taskParamProp ?taskParamClass ?taskParamPosition
WHERE {
?subClass rdfs:subClassOf tasks:Parameter .
?taskParam rdf:type ?subClass .
?taskParam rdf:type ?objType .
?taskParam tasks:nameIs ?taskParamName .
?taskParam tasks:propertyWithValueIs ?taskParamProp .
?taskParamClass tasks:parameterIs ?taskParam .
?taskParam tasks:parameterPositionIs ?taskParamPosition .
}
ORDER BY ?taskParamClass ?taskParamProp ?taskParamPositionNote that this query returns at least 2 results for every Task Parameter because each will implement optional or required parameter rdf types.
The following rules are used when generating Task parameters:
If task parameters already exist in the IRIS OWL files for a particular class, then no task parameters are generated for that class. However, the task verifies that the task parameter in the ontology matches the ontology usage documentation and fails if they don't match.
The task parameter will implement InputParameter and/or OutputParameter depending on whether or not it is marked as susch in the ontology usage documentation. Currently there are no task parameters which are both input and output, but it is possible to mark them so.
The Parameter instance will also implement one of RequiredParameter or OptionalParameter from inspecting the usage documentation. Therefore all Parameter instances will implement at least 2 rdf types.
The nameIs slot is formed by taking the local name of the ontology property and removing a trailing "is" or preceding "has".
parameterPositionIs is an integer index (1 based) which is the srt order of the property in alphabetical ordering in the set of all task parameters for the task parameter's class.
positionIs is set to the same as parameterPositionIs.
propertyWithValueIs refers to the ontolgoy property for the task parameter.
For the ontology class from the ontology usage documentation. the annotation property parameterIs is set to refer to the Parameter instance.
A Task Parameter instance is generated for every Task that it is used in. For example, the property hasEmail has many tasks in its domain. Therefore tis is likely (according to ontology usage documentation) that there is more than one task parameter instance that refers to this property, each added to a unique ontology class.
To restate the above, it is never the case that a Task OWL class contains 2 Task Parameter instances that point to the same ontology property. The task parameter generation task ensures this by merging ontolgoy usage documentation.When merging, if one ontology usage documentation indicates it consumes automation and the other does not, the one that consumes automation is used.
The following consistency checks are also performed for all task parameters and the build fails:
If a class refers to task parameter that no longer exists
If a task parameter is set on something that is not an OWL class
If a task parameter refers to a property that doesn't exist because its name changed, or the property is not set. A check is also done to make sure the property has a range set (regression from previous error where tasks:hasDocumentInputData did not have a range set).
There isn't 2 task parameter instances with the same position. Also, there is a position set on every task parameter.
The following can be used to test the various checks work (note: each test assumes a clean taskparams.owl):
Open up iris/data/mailApp.owl and scroll to the DeleteEmail event class doc, destEmailFolder property. Change isOutputProperty to isInputProperty. Then re-run the task generation task. It will fail because the ontology tak parameter does not match the ontology usage documentation.
Open taskparams.owl in Protege, and delete the "propertyWithValueIs" value for one of the parameters (ex: "Parameter-AddCalendarEntry-hasCalendarEvent"). Then re-run the task parameter generation and it will fail.
Open taskParams.owl in a text editor and find one of the parameterIs references, and then name it to something that doesn't exist (ex: add a 'Bogus' on the end). Task parameter generation will fail. For example, if the following is set the task parameter generation will fail:
<rdf:Description rdf:about="http://calo.sri.com/tasks#OpenEmailWindow">
<j.0:parameterIs rdf:resource="http://calo.sri.com/bogus#BogusParam" />
</rdf:Description>Edit taskParams.owl in Protege, find a class that has 2 or more parameterIs annotations and set 2 of them to have the same position. Task parameter generation will fail.
Edit taskParams.owl in Protege, find a class that has 2 or more parameterIs annotations and remove one of the parameterIs assertions. Task parameter generation will fail.
After generating task params, copy them to iris/data and open/save iris/data/taskParams.owl in Protege (without making changes). make sure the task param generator task runs afterwards. This is a sanity check that task param generation still works after Protege writes the file. it also tests that validation tests work on the persisted task params.
Finally, the following beanShell script can be used to test task parameter POJO output and shows how POJOs can be used to inspect task parameters:
import com.sri.iris.store.pojo.*;
import com.radarnetworks.som.*;
SemanticClass cls = kbc.getSemanticClass(ISendEmailPojo.URI);
Object[] taskParams = cls.getAllValues("http://calo.sri.com/tasks#parameterIs");
StringBuffer debug = new StringBuffer();
debug.append("Got ").append(taskParams.length).append(" task parameters\n");
for (int i = 0; i < taskParams.length; i++) {
IParameterPojo param = (IParameterPojo)taskParams[i];
debug.append("Task Parameter:\n");
debug.append(" Name: ").append(param.getNameIs()).append("\n");
// Note that the ordering is not guaranteed which is why each parameter has a position
debug.append(" Position: ").append(param.getParameterPositionIs()).append("\n");
debug.append(" Property: ").append(param.getPropertyWithValueIs().getURI()).append("\n");
if (param instanceof IInputParameterPojo) {
debug.append(" Parameter is Input").append("\n");
}
if (param instanceof IOutputParameterPojo) {
debug.append(" Parameter is Output").append("\n");
}
if (param instanceof IRequiredParameterPojo) {
debug.append(" Parameter is Required").append("\n");
}
if (param instanceof IOptionalParameterPojo) {
debug.append(" Parameter is Optional").append("\n");
}
}
debug.toString();Note that there are no POJOs for class instances so the task parameter annotations must be retrieved directly from the SemanticClass instance. Once retrieved, the task parameter instances are exposed as POJOs.
A Binding Path is a definition of a traversal from one ontology instance to a piece of data along a path of properties and instances branching from that instance. Each segment in the traversal may contain constraints which are used to narrow the branching.
For example, a Binding Path may be "the work street address for a Person". The starting instance is a Person and the final piece of data is an email address string. To get from a Person to an address string is a complicated path in the ontology. A Person "contactInfoIs" WorkContactEntry and then "malingAddressIs" MailingAddress which has "addressStreetIs" (the work mailing address, or final piece of data in the Binding Path).
Why use Binding Paths? Binding Paths are used for the following reasons:
For querying the ontology it is typical to make requests such as "Get all mailing addresses for a Person". The upper ontology is complex in nature, and binding paths are a way to "flatten" the complex ontology. Binding paths are typically used in this way for query-only purposes, and they can be used in SPARQL queries (they inherit from owl:DatatypeProperty and owl:BindingPathProperty).
The current form data views in IRIS only support a flat view, and so binding paths are used.
Binding paths are also used to set data, particularly in the form data views. This is not a typical use case, but it is convenient and IRIS will create intermediary objects along the path if they do not exist. For example, when setting the work mailing address of a person, the clib:WorkContactEntry and clib:MailingAddress will be created if they do not exist.
To rephrase statements above, a BindingPathProperty can act as a "super-property" that spans a graph traversal. This a useful feature to have when querying, and complex queries can be built using binding paths. Also note that binding paths may refer to other BindingPathProperties as well.
Binding paths are defined in the file iris/data/bindingPaths.owl. There is an ontology used to describe binding paths and that ontology exists in iris/data/irisOntology.owl. Note that bindingPaths.owl is written to be readable in any text or XML editor. This file can also be loaded into Protege. A BindingPath has the following characteristics:
Just like OWL properties, a binding path has a unique URI. For example, "http://iris.sri.com/irisOntology#bp_contact_work_email"
The set of classes that the binding path may start from (ex: Person).
A binding path can be marked query only, meaning it cannot be used to set data.
A segment is a single traversal of a property on an instance.
Note that ranges are not defined for binding paths. This is because IRIS automatically reports the binding path range as the range of the last traversal (at runtime). However, binding paths are distinguished as being object or datatype properties.
Each segment has the following:
The property to be traversed.
Note that this implies owl:Full. IRIS does not support all of owl:Full. However, it does support properties that have OWL classes/properties in the range.Restrictions enforce rules on the binding path. This is include cardinality of 1 or infinite when auto-creating paths, and also an optional class restriction on the instance.k
The next segment in the path
The following are valid restrictions (Note that not all these restrictions apply to all segment, see Simplification Rules for details):
Optional, specified when a segment's ontology property is an instance of owl:ObjectProperty. The RestrictiveClass points to an owl:Class for the type of data expected to be contained in the property.
Specifies:
The minumum number of occurences ofdata values for the property.
The maximum number of occurences ofdata values for the property.
The Cardinality is used when creating data in the binding path. Note that this definition of Cardinality differs from generic cardinality on an owl property (specific to an owl class) in that this specifies Cardinality for the ontology usage which this binding path is for, which may be constrained with a subset of the Cardinality for the owl class and property. The default Cardinality is min range 0, and no max range (infinite) if no Cardinality is found for the segment. Note that "infinite" is represnted by Java's Integer.MAX_VALUE = 0x7fffffff. Also note that only single and infinite cardinality is supported.
Cardinality is used when setting data and auto-creating instances. IRIS needs to know when to branch.
Specifies that another property must equal a certain value. Only data type string is supported. The SiblingPropertyEquals specifies:
The ontology property which is a sibling property on the instance
The value which the ontology property must be equal to (equality is case sensitive).
As an example, consider a query-only binding path which retrieves all street addresses for a person, organization, etc. including home, work, etc:
<iris:BindingPathDatatypeProperty rdf:about="http://iris.sri.com/irisOntology#bp_contact_all_street_address">
<rdfs:domain>
<owl:Class>
<owl:unionOf rdf:parseType="Collection">
<owl:Class rdf:about="http://calo.sri.com/core-plus-office#Project"/>
<owl:Class rdf:about="http://calo.sri.com/core-plus-office#Task"/>
<owl:Class rdf:about="http://calo.sri.com/core-plus-office#Team"/>
<owl:Class rdf:about="http://calo.sri.com/core-plus-office#Person"/>
<owl:Class rdf:about="http://calo.sri.com/core-plus-office#Organization"/>
</owl:unionOf>
</owl:Class>
</rdfs:domain>
<iris:hasBindingPath>
<iris:BindingPath>
<doc:hasDocumentation>
<doc:Documentation>
<doc:hasMarkedUpContent>Getting all street addresses:</doc:hasMarkedUpContent>
<doc:hasAuthorFullName>Rich Giuli</doc:hasAuthorFullName>
</doc:Documentation>
</doc:hasDocumentation>
<iris:isQueryOnly rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</iris:isQueryOnly>
<iris:hasBindingPathElement>
<iris:BindingPathElement>
<iris:hasOntologyProperty rdf:resource="http://calo.sri.com/core-plus-office#contactInfoIs"/>
<iris:hasOntologyPropertyRestriction>
<iris:RestrictiveClass>
<iris:hasRestrictiveClass rdf:resource="http://calo.sri.com/core-plus-office#ContactEntry"/>
</iris:RestrictiveClass>
</iris:hasOntologyPropertyRestriction>
<iris:hasBindingPathElement>
<iris:BindingPathElement>
<iris:hasOntologyProperty
rdf:resource="http://calo.sri.com/core-plus-office#mailingAddressIs"/>
<iris:hasOntologyPropertyRestriction>
<iris:RestrictiveClass>
<iris:hasRestrictiveClass
rdf:resource="http://calo.sri.com/core-plus-office#MailingAddress"/>
</iris:RestrictiveClass>
</iris:hasOntologyPropertyRestriction>
<iris:hasBindingPathElement>
<iris:BindingPathElement>
<iris:hasOntologyProperty
rdf:resource="http://calo.sri.com/core-plus-office#addressStreetIs"/>
</iris:BindingPathElement>
</iris:hasBindingPathElement>
</iris:BindingPathElement>
</iris:hasBindingPathElement>
</iris:BindingPathElement>
</iris:hasBindingPathElement>
</iris:BindingPath>
</iris:hasBindingPath>
</iris:BindingPathDatatypeProperty>In order to simplify the interpretation of binding paths, some simple rules are established:
There can only be one segment with a Cardinality that has max range greater than 1.
No restrictions or rules are allowed for the last segment except for Cardinality. The other restrictions and rules apply to creation or traversal only, which isn't relevant for the last segment (because the there is nowhere else to traverse and the data is already created).
Only cardinality 1 or infinite is suppported.
Data creation is complex. There are some special cases that can be handled to optimize the creation, which are listed below. In the operations are specified various return values which are described here:
A class containing the following information:
The instance which was replaced, or null if no instance was replaced.
A list of object instances that leads from instance to data. This is the "branch" which was added, removed or retrieved. This value is returned even when no data is set (ex: from an Assert). In this case the path returned is the path that already exists.
Only one segment: In the case there is only one segment, here are the following operations that can modify data and how they are applied:
First examines the Cardinality of the segment. If the Cardinality's max range has not been reached, a new value is added for the property. If the Cardinality's max range has been reached, a property is replaced according to the rules defined previously.
Examines all the properties and if the property is not already set, an Add is performed.
Note that in IRIS there can never be a duplicate triple. Therefore if Add is called twice with the same value, only one value is added. So it follows that an Assert operation is the same as an Add operation.Gets all properties for the segment, and changes them to be the input data being set.
Note that in IRIS there can never be a duplicate triple. Therefore if Add is called twice with the same value, only one value is added. So it follows that an ReplaceAll operation is the same as a RemoveAll followed by an Add.Removes all properties for the segment.
A RemoveAll(instance) followed by an Add(data, instance)
Retrieves all values for the property. If there is more values than the Cardinality's max range, an error is logged (but no corrective action taken) and the first N values are returned where 'N' is the max cardinality (first being arbitrary and undefined).
Retrieves the first (arbitrary) value from GetAllValues(instance).
The following algorithm is used in the operations that follow:
This algorithm is called only if a BindingPath cannot be traversed because not all the segments exist. Constraints are:
startSegmentIndx is the segment where realization starts
startInstance is where the traversal starts
If startSegmentIndx is 0, then startInstance must be in the domain of the BindingPath
If startSegmentIndx is > 0, startInstance must implement the RestrictiveClass of the *previous* segment.
startSegmentIndx cannot be the final segment.
Algorithm:
If the binding path is query only nothing is done and the operation is aborted.
Otherwise, segments are examined walking forward from startSegmentIndx. If a segment doesn't exist, a instance is created using the restrictions and rules to create the instance. If a segment does exist, and the max cardinality has not been reached, then a new instance is also created. This is to required to handle cases such as the following:
Suppose there is the binding path "Person authors EMailMessage" and also "Person authors Document". Each of these binding paths starts with "Person plays Author has in-event Write", but have different endings. If it is asserted that a Person authors a Document, when it is first asserted that a Person authors an EMailMessage the binding path will not resolve to anything, but Write events will exist for the Author. It is most appropriate in these cases to create a new Write event (when max cardinality has not been reached).
Note: If when walking an existing segment and more than one value is found that matches the restrictions, but the Cardinality max range of the segment is 1, then this is an error. An error is logged (but no corrective action is taken), and the first (arbitrary) instance is used.
When creating the instance for a segment, the SiblingPropertyEquals is checked (if it exsts),l by getting all current values for the segment and then retrieving the specified one. If multiple are found, the fist is used. If none are found, this implies creation of the segment.
Finally, for the last segment the input data is simply set on the current instance.
Single Path: In the case that there is more than one segment, and all the segments have Cardinality maxRange 1, the following operations can be applied:
Equivalent to Assert(data, instance).
Calls RealizeBindingPath(data, instance, 0) if the binding path is not realized. If the binding path is realized, the Assert operation examines the final instance. If it differs from the input value, then RemoveAll(instance) is called followed by RealizeBindingPath(data, instance, 0). Note that RemoveAll is performed in order to properly remove the next-to-last instance given its creation scope. Also note that if the instance in the binding path is the same then nothing is done. Equality for checking this is determined by comparing primitive types in the usual way, case sensitive for strings, and comparing object uris for objects.
Equivalent to Assert(data, instance).
the last segment is removed instead of the next-to-last segment. For example, if a person's work email were instance specific (which it isn't, but this is just an example), then the hasAddressString value would be removed.
Note that RemoveAll does not delete the value removed if the value is an object, and it does not delete any other instances for other segments (Note: delete is a sensitive issue and is currently not handled well in IRIS).A RemoveAll(instance) followed by an Add(data, instance).
If the binding path is not realized fully until segmentIndx, return null. Otherwise each segment is traversed. If there is more values than the Cardinality's max range for any segment (which is 1 in this case), an error is logged (but no corrective action taken) and the first value is used (first being arbitrary). The operation stops at the endSegmentIndx specified, which is the number of the last segment to stop at. For example, if stopping at segment 1 (0 based index) when getting a person's email address, that would correspond to the 2nd segment which is an instance of ContactRecordSubAddress.
Retrieves the first (arbitrary) value from GetAllValues(instance, endSegmentIndx). Note that OWL/RDF does not impose ordering so the first value is arbitrary.
Equivalent to GetAllValues(instance, endSegmentIndx), where endSegmentIndx is the last segment index.
Equivalent to GetValue(instance, endSegmentIndx), where endSegmentIndx is the last segment index.
Branching Path: In the case there is more than one segment, and the Cardinality max range is 1 on all segments except for one segment, this is the most general binding path and the following operations are defined:
Let branchSegmentIndx point to the segment index where the Cardinality max range is > 1.
Calls GetAllValues(instance). If the return value is null, RealizeBindingPath(data, instance, 0) is called; otherwise an add is performed as follows:
If branchSegmentIndx is the last segment, then it is possible from the Simplification Rules that the Cardinality is > 1 but < infinite. If this is the case and the maximum cardinality is reached, then a value is replaced using the guidelines defined above in the section Simplification Rules. If max cardinality has not been reached, GetAllValues(instance, nexToLastSegmentIndx) is called, and for each of these instances data is added as a value for the property defined in the last segment (because cardinality is > 1 at the last segment).
Note that in IRIS there can never be a duplicate triple in the same knowledge base. Therefore if Add is called twice with the same value, only one value is added.If branchSegmentIndx is not the last index, the cardinality at the segment before branchSegmentIndx must be 1 (Simplification Rules), so it is possible to get that one instance and branch from there. Calling RealizeBindingPath(data, GetValue(instance,branchSegmentIndx-1), branchSegmentIndx) will branch from the branching point and add the new data value. Note that a -1 may be passed to GetValue, and that case is handled below.
Note that this case is not alwasy desireablr; normally an Assert(data, instace) should be called instead. For example, given the binding path "Person authors EmailMessage", if you were perform an add operation twice in a row that the Person authors the same EMailMessage, 2 branches are create using the Add operation, but the Assert operation recognizes the first branch and doesn't add a second branch.Note that this operation may be optimized to simultaneously loop through GetAllValues and perform the add if GetAllValues succeeds (instead of having to do 2 operations
Calls GetAllValues(instance). If the return value is null or the input value is different from all return instances, Add(data, instance) is called; otherwise nothing is done (if any of the return instances is equivalent the input value). Equality for checking this is determined by comparing primitive types in the usual way, case sensitive for strings, and comparing object uris for objects.
Calls GetAllValues(instance, nextToLastSegmentIndx). If null is returned, then nothing is done. Otherwise for each return instance all values for the final property specified in the last segment are replaced with 'data'.
Note that in IRIS there can never be a duplicate triple in the same knowledge base. Therefore if Add is called twice with the same value, only one value is added. So it follows that after a ReplaceAll the cardinality of the "leaves" of the binding path will always be 1 event if the cardinality was > 1 before the ReplaceAll.Calls GetAllValues(instance, nextToLastSegmentIndx). If null is returned, then nothing is done. Otherwise tthe last segment property is removed on all instances. For example, if a person's work email were instance specific (which it isn't, but this is just an example), then the hasAddressString value would be removed.
Note that RemoveAll does not delete the value removed if the value is an object, and it does not delete any other instances for other segments (Note: delete is a sensitive issue and is currently not handled well in IRIS).A RemoveAll(instance) followed by an Add(data, instance).
If the binding path is not realized fully until segmentIndx, return null. Otherwise each segment is traversed. When a segment is traversed, all objects from the previous traversal are traversed in unison (with the very first traversal set being the input instance). If at a particular segment traversal for a single instance there are more values than the Cardinality's max range, an error is logged (but no corrective action taken) and the first value(s) are used (first being arbitrary).
If segmentIndx is -1, return instance; otherwise retrieves the first (arbitrary) value from GetAllValues(data, endSegmentIndx).
Equivalent to GetAllValues(instance, endSegmentIndx), where endSegmentIndx is the last segment index.
Equivalent to GetValue(instance, endSegmentIndx), where endSegmentIndx is the last segment index.
IRIS exposes the following APIs for binding paths:
Accessor for retrieving all known bindings. IRIS has 2 binding factory implementations:
Reads binding paths from a knowledge base.
Used to read bindings from disk. Because it can take a while to load bindings from the ontology, IRIS writes them to disk at build time and then reads them back in at runtime. This improves startup time. This loading is done in com.sri.iris.jena.factory.KnowledgeBaseFactoryLoader :
if (!BindingFactory.isInitialized()) {
new SerializedBindingFactoryInitializer(IThingPojo.class.getClassLoader().
getResource("com/sri/iris/store/data/ontology_bindings_map.ser")).initialize();
}Specification of a binding path. This is the spec read from the ontology.
The JENA graph that interprets binding paths. This graph also deduces ranges. This graph is used at the highest level so that the transitive reasoner and any other reasoners are applied when traversing segments off binding paths.
JENA integration of binding paths. It is possible to use JENA APIs to detect binding path properties. For example, if "property" is an instance of com.hp.hpl.jena.rdf.model.Property:
if (property.canAs(BindingPathOntProperty.class)) {
BindingPathOntProperty prop = (BindingPathOntProperty)property.as(BindingPathOntProperty.class);
BindingPathProperty propDef = prop.getBindingPathProperty();
// ....
}Ontology interfaces in IRIS exist on different levels:
com.sri.iris.jena.factory.IKnowledgeBaseFactory is the highest level providing access to all knowledge bases
POJOS are the programming API into the ontology which are built on top of the SOM
SOM or Semantic Object Model is the intermediary layer which abstracts access to the underlying ontology implementation.
Ontology Implementation is at the lowest layer and exists as the implementation classes for the SOM interfaces. Currently this is JENA model and set of JENA graphs.
In the next sections IKnowledgeBaseFactory is documented first followed by the SOM and then POJOs. The SOM is discussed before POJOs because POJOs are built on top of the SOM.
com.sri.iris.jena.factory.KnowledgeBaseFactoryLoader identifies the plugin which loads and registers the IRIS IKnowledgeBaseFactory, IObjectStore, and IUser services.The bean "KnowledgeBaseFactoryLoader" in iris/bin/config/kb.xml loads these services.
An IRIS plugin accesses the KnowledgeBase by using the following IRIS services:
A convenience interface into the knowledge base. This is a simplified interface, and it is recommended that the IKnowledgeBaseFactory interface be used for more complex interaction. In particular, the IObjectStore interface has the following methods for interacting with the ObjectStore:
KnowledgeBaseConnection getKnowledgeBaseConnection() retrieves the connection to the KnowledgeBase. This is the most useful interface to the KNowledgeBase and is detailed more below.
SemanticObject create(String classUri) and it's sibling method which takes an instance uri is used to create a new instance of an ontology class uri. The SemanticObject returned is accessible via the SOM API and can also be cast to an instance of a POJO.
The main interface into the IRIS knowledge base. This IRIS service allows access to KB separation graphs. In particular, the createKnowledgeBase methods allow access to separate knowledge bases. See the documentation below on KB separation for more information.
Allows access to a specific knowledge base
Allows direct access to ontology objects in a knowledge base.
Allows access to the login name of the user and the User POJO instance which contains the user's login name and password. Note that for a more decriptive user information object, the Harvester service should be used (see below).
A service used for low-level normalizing and creating data within the ontology. Some useful methods of this service are:
IPersonPojo getUserPerson() Retreives the Person POJO. Note that the IPerson POJO differs from the IUserPojo specified above in that IUserPojo is a lightweight wrapper for just login information, whereas this Person POJO is the encapsulation of all the user's information in IRIS.
... getXNormalizer() The normalizers exist in order to find objects so that they will not be created more than once. There are normalizers for file, email and contact information.
... getXCreator() The creators are used to facilitate creating complex data objects in the ontology. Usually a creator will take data from an external form and put it into the ontology. For example, the email creator takes an IEmailData instance which specifies an email and then copies the data into the ontology. Creators exist for email, contacts and files.
KnoweldgeBase separation, or KB separation is the ability to segregate data in IRIS but still allow queries to work across the aggregated data. KB separation involves the ability to define multiple graphs--to which data will be written--and custom Knowledge Bases--a view of graphs to be read and an explicit writable graph. KB Separation achieves the following goals:
Allows the creation of multiple persistent Jena Graphs. Graphs are the underlying data structures that allow the reading and writing of RDF data. A graph is the most granular unit of data separation within IRIS. An API is provided for IRIS plugins to use a custom Jena Graph, or to predefine a custom IRIS JDBM graph with or without full text indexing (more details on this below).
Allows the configuration of a custom KnowledgeBase by specifying a primary Graph where data will be written and a collection of named graphs where data can be read.
The ability to take any POJO transactional object and commit changes to another knowledge base.
Top-level queries use Jena's MultiUnion approach to query all Graphs within any KnowledgeBase's view.
Long term goals of KB separation are:
The ability to take manage learned information from CALO (RQ-CORE-PLUGIN). If learned information is stored in separate KBs, then taking those KBs away will effectively remove most learned data.
The ability to detect from a unioned query which results come from a specific KB. This is supported using the GRAPH syntax with SPARQL.
Static configuration of KBs is specified in iris/bin/config/kb.xml The following property allows the separate Graphs to be configured, and is specified in the KnowledgeBaseFactoryLoader config:
<property name="jdbmGraphs">
<description>
Specifies KB separation graphs in IRIS. Each GraphConfig gives a directory name and specifies whether
or not full text indexing is supported. The map maps the KB names t the graph. These KB names can then
be used in the IKnowledgeBaseFactory API to retrieve a KnowledgeBase instances with a specific graph
that
is writable.
</description>
<set>
<bean class="com.sri.iris.jena.factory.GraphConfig">
<property name="graphName" value="mainkb"/>
<property name="dirName" value="jenadb"/>
<property name="supportsFullTextQuery" value="true"/>
<property name="graphData">
<map/>
</property>
</bean>
<bean class="com.sri.iris.jena.factory.GraphConfig">
<property name="graphName" value="mailkb"/>
<property name="dirName" value="maildb"/>
<property name="supportsFullTextQuery" value="true"/>
<property name="graphData">
<map/>
</property>
</bean>
<bean class="com.sri.iris.jena.factory.GraphConfig">
<property name="graphName" value="filekb"/>
<property name="dirName" value="filedb"/>
<property name="supportsFullTextQuery" value="true"/>
<property name="graphData">
<map/>
</property>
</bean>
<bean class="com.sri.iris.jena.factory.GraphConfig">
<property name="graphName" value="contactkb"/>
<property name="dirName" value="contactdb"/>
<property name="supportsFullTextQuery" value="true"/>
<property name="graphData">
<map/>
</property>
</bean>
<snip />The above indicates that 3 persistent graphs should be created. A "graph" is a store of RDF triples. In this case, 3 graphs are created: "mainkb", "mailkb" and "contactkb". The above configuration specifies the locations of the graphs, giving folder names. In order for clients to create a KnowledgeBase view of these graphs, the method createKnowledgeBase is used.
Note that static XML-based configuration should only be used when:
The KB is core to the functionality of IRIS
The configuration is local to some custom version of IRIS that will be maintained
The KB is not part of an IRIS plugin that may or may not run with IRIS (ie, it needs to be core to the functionality of IRIS or in a custom IRIS)
This also means that kb.xml should not be used to statically specify KB separate graphs for CALO Desktop. For CALO Desktop the API should be used by plugins to ensure that KBs are created (see next section).
iris/bin/config/kb.xml is also used to define memory graphs:
<property name="memoryGraphs">
<description>
These graphs reside in memory. Any changes made to the graphs here do not persist unless the graph is
set to persist.
</description>
<map>
<entry key="events">
<bean class="com.sri.iris.jena.factory.MemoryGraphConfig">
<property name="purgeTime" value="1800"/>
<property name="persistGraph">
<!-- Note querying the hybrid graph also queries the persist graph -->
<null/>
</property>
</bean>
</entry>
</map>
</property>Memory graphs contain transient data. To avoid running out of memory the transient data is purged. Triples older than purgeTime (seconds) are removed. If persistGraph is defined, then purged triples are written to the persist graph. To define a persist graph, define an instance of a graph config. For example:
<entry key="events">
<bean class="com.sri.iris.jena.factory.MemoryGraphConfig">
<property name="purgeTime" value="1800"/>
<property name="persistGraph">
<!-- Note querying the hybrid graph also queries the persist graph -->
<bean class="com.sri.iris.jena.factory.GraphConfig">
<property name="graphName" value="peventskb"/>
<property name="dirName" value="peventsdb"/>
<property name="supportsFullTextQuery" value="true"/>
<property name="graphData">
<map/>
</property>
</bean>
</property>
</bean>
</entry>Note that adding persistence for the memory graphs will
reduce performance drastically.Memory graphs use JENA's faster graph mem model by default. See the runtime parameter for toggling this. To create a memory graph programmatically, use the class com.sri.iris.jena.factory.IrisGraphMemFactory. For example:
IGraphMem memoryGraph = IrisGraphMemFactory.newMemGraph();
memoryGraph.setReadOnly(false);
memoryGraph.setTypeRestrictive(true);It is possible to make a memory graph read-only and type restrictive. A type restrictive graph disallows any triple addition or deletion if the memory graph does not contain the rdf:Type declaration for the subject in question. This is turned on by default and is a safety mechanism to prevent persistent data accidently being written to a transient memory graph. In other words, it safegaurds against accidental loss of data.
com.sri.iris.jena.factory.IGraphCreator is an IRIS bean and has two methods for creating storage graphs:
Creates a Graph on disk that will persist across multiple runs of IRIS, backed by a JDBM BTree.
Creates a Graph in memory that can be used for temporary storage and will be destroyed when IRIS shuts down
A JDBM graph is a graph based on a JDBM database which is a fast lightweight persistent storage engine for Java. The bean has the name "com.sri.iris.jena.factory.IGraphCreator" and implements the interface IGraphCreator. This interface allows creation of a JDBM graph given an IGraphConfig instance. The IGraphConfig allows the directory name of the separate graph to be specified. Note that the directory name is not a full path but instead a single folder name (ex : "mydb"). This is enforced because all IRIS database files are stored relative to a common root directory that can be changed. See the example below for more information on creating a JDBM graph.
Graphs created with IGraphCreator can be used to assemble KnowledgeBases through IKnowledgeBaseConfiguration instances and the IKnowledgeBaseFactory.
IKnowledgeBaseFactory creates KnowledgeBase instances that can be backed by various Graphs for reading and writing data. Graphs can be made public to the IKnowledgeBaseFactory and given a name using the addGraph(String name, Graph graph) method. If a Graph is named, it is automatically made public and can be used by any component while configuring a KnowledgeBase. If a Graph is to remain private, do not attach a name to it by calling addGraph; the private unnamed Graph can still be used as the primary, writable graph for any KnowledgeBase.
In order to get KnowledgeBaseConnection and retrieve a configured KnowledgeBase, first declare the IKnowledgeBaseFactory as a dependency for your bean (described later) Then, create an IKnowledgeBaseConfiguration to describe the KnowledgeBase that you want to create. An IKnowledgeBaseConfiguration specifies:
the primary Graph to which incoming data to the KnowledgeBase will be written
a list of Graph names to include in the queries to the KnowledgeBase, defaulting to all public named graph, present and future
a list of Graph names to exclude while querying the KnowledgeBase, defaulting to none
An example configuration follows, describing a KnowledgeBase that writes to the Graph named "mailkb" and reads from all Graphs except the Graph named "events":
1 IKnowledgeBaseFactory knowledgeBaseFactory;
2 IKnowledgeBaseConfiguration kbConfig = new KnowledgeBaseConfiguration(knowledgeBaseFactory, "mailkb");
3 HashSet<String> graphExclusions = new HashSet<String>();
4 graphExclusions.add("events");
5 kbConfig.setExclusionNames(graphExclusions);
6 KnowledgeBase kb = knowledgeBaseFactory.createKnowledgeBase(kbConfig);Alternatively, a component may want to create a KnowledgeBase backed by its own private Graph that includes all Graphs except the "mailkb" Graph:
1 IGraphCreator graphCreator;
2 IKnowledgeBaseFactory knowledgeBaseFactory;
3 IKnowledgeBaseConfiguration kbConfig = new KnowledgeBaseConfiguration(graphCreator.createMemoryGraph());
4 HashSet<String> graphExclusions = new HashSet<String>();
5 graphExclusions.add("mailkb");
6 kbConfig.setExclusionNames(graphExclusions);
7 KnowledgeBase kb = knowledgeBaseFactory.createKnowledgeBase(kbConfig);Simple, all-inclusive configurations can be obtained with legacy convenience methods. They allow for the creation of a KnowledgeBase that will read from all public named graphs and write to one specified graph. The following code retrieves a KnowledgeBaseConnection instance backed by a graph named "mykb":
1 KnowledgeBaseConnection kbc = knowledgeBaseFactory.createKnowledgeBase("mykb").getConnection();The same is also possible without using a public, named graph as the writable graph. The following call retrieves a KnowledgeBaseConnection that writes to the graph "myGraph" which is of type Graph:
1 IGraphCreator graphCreator; 2 Graph myGraph = graphCreator.createMemoryGraph(); 3 KnowledgeBaseConnection kbc = knowledgeBaseFactory.createKnowledgeBase(myGraph).getConnection();
Another set of legacy convenience methods is provided to allow for the creation of a KnowledgeBase that will only read from the single Graph to which it writes. These methods are createKnowledgeBase(String graphName, boolean includePublicGraphs) and createKnowledgeBase(Graph primaryGraph, boolean includePublicGraphs). It may be preferable to use these in order to improve performance. For example, if the application knows it only needs to access a particular KnowledgeBase. Passing a "false" as the second parameter of these APIs should be used with extreme caution and only under circumstances where it is necessary and will not cause data integrity problems.. For example:
1 KnowledgeBaseConnection kbc = knowledgeBaseFactory.createKnowledgeBase("mykb", false).getConnection();or
1 KnowledgeBaseConnection kbc = knowledgeBaseFactory.createKnowledgeBase(myGraph, false).getConnection();
Both of these calls are equivalent to the following IKnowledgeBaseConfiguration:
KnowledgeBaseConfiguration kbConfig = new KnowledgeBaseConfiguration(myGraph);
kbConfig.setExclusionNames(IKnowledgeBaseConfiguration.ALL_GRAPHS);Note the following regarding the return KnowledgeBase instances from these APIs:
For a given configuration, the same KnowledgeBase instance will be returned if it is called more than once.
Each connection writes to the specified primary graph. When querying the connection, however, all included graphs are queried using a MultiUnion.
Because all knowledge bases are queried, to guarantee that writes will always go to the KnowledgeBaseConnection desired, the transactional API must be used to commit changes to a specific knowledge base.
When retrieving a SemanticObject, any writes made to that SemanticObject go to the KnowledgeBase in which the rdf:type is defined for the object. Because of this default behavior, transactional POJOs or the "inConnection" method in SemanticObject must be used to guarantee writes to a specific KnowledgeBase.
As mentioned above, to gaurantee data will be commited to the right KnowledgeBase requires using Transactional POJOs. A Transactional POJO is a POJO that implements the TransactionalSemanticObject interface. This interface has 3 methods on it:
determines if the POJO has been modified.
commits any pending changes to a specific KnowledgeBaseConnection.
reverts any pending changes
Note that the word "transactional" is used loosely here. This does not indicate a true ACID transaction for all pending changes. It simply means all changes will be committed to a specific KnowledgeBaseConnection.
Given a POJO, there are method to create transactional POJO instances from that POJO. For example, the following code creates a transactional POJO for a Person POJO, sets the work email and then commits the change to a specific KnowledgeBaseConnection:
1 IPersonPojoX transactionalPojo = existingPerson.newTransactionalPersonPojo(); 2 transactionalPojo.setContactWorkEmail(emailAddress.getRawAddress()); 3 transactionalPojo.commit(kbc);
Note also that it is possible to use transactional POJOs in the SOM layer instead of using POJOs. Given a SemanticObject instance, the following code retrieves a TransactionalSemanticObject:
1 TransactionalSemanticObject transactionalSemanticObject = SemanticObjectUtilities.newTransactionalSemanticObject(obj);
where "obj" is of type "SemanticObject.
Creating a KnowledgeBase or KnowledgeBaseConnection from a named static graph is simple and straightforward; simply use the IKnowledgeBaseFactory API. However, creating from a JDBM graph is not as straightforward. This example demonstrates the configuration and implementation for doing this.
In this example, the KnowledgeBase "mokb" is being created and used. It is assumed the reader is familiar with how to configure IRIS. Here is the configuration for the IRIS plugin, with the necessary configuration parameters for a JDBM graph:
1 <?xml version="1.0" encoding="UTF-8"?> 2 <!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd"> 3 <beans> 4 <bean class="com.sri.desktop.plugins.mokb.MOKBPluginImpl" 5 name="com.sri.desktop.plugins.mokb.IMOKBPlugin" 6 init-method="startup" destroy-method="shutdown" lazy-init="true"> 7 <description> 8 Plugin providing MOKB functionality in Desktop. 9 This document was created by using the IRIS Plugin SDK. See: 10 iris/doc/howtos/howtoConfigure.html 11 </description> 12 <property name="knowledgeBaseFactory"> 13 <description> 14 This is the main interface to the IRIS KB. The MOKB plugin can use this in order to add a new 15 graph in KB separation and access knowledge bases in IRIS. Once a connection to the graph is 16 obtained, the Jena model can be retrieved. 17 See the document iris/doc/howtos/howtoOntology.html for more information. 18 </description> 19 <ref bean="com.sri.iris.jena.factory.IKnowledgeBaseFactory" /> 20 </property> 21 <property name="graphCreator"> 22 <description> 23 IGraphCreator is used to create a Jena Graph from an IGraphConfig. 24 </description> 25 <ref bean="com.sri.iris.jena.factory.IGraphCreator" /> 26 </property> 27 <property name="mokbGraphConfig"> 28 <description> 29 The GraphConfig specifies the directory name for the database files and whether or not the graph 30 supportd full text query. THis graph is our own custom JDBM graph. We also have the ability to add 31 any Jena Graph as a seaprate KB in IRIS, but you have to use the IKnowledgeBaseFactory API directly. 32 See iris/doc/howtos/howtoOntology.html for more information. 33 </description> 34 <bean class="com.sri.iris.jena.factory.GraphConfig"> 35 <property name="dirName" value="mokbdb" /> 36 <property name="supportsFullTextQuery" value="true" /> 37 </bean> 38 </property> 39 <property name="mokbGraphName"> 40 <description> 41 This is the name of the graph for mokb. This name is used whenever a component needs to identify the 42 MOKB graph. 43 </description> 44 <value>mokb</value> 45 </property> 46 </bean>
See the comments above for more information. Note that an instance of IGraphConfig is specified in the configuration. It is recommended to do this.
The following is source code for this bean:
1 package com.sri.desktop.plugins.mokb;
2
3 import com.sri.iris.jena.factory.IKnowledgeBaseFactory;
4 import com.sri.iris.jena.factory.GraphConfig;
5 import com.sri.iris.jena.factory.IGraphCreator;
6 import com.sri.desktop.plugins.oaakb.IOAAKB;
7 import com.hp.hpl.jena.graph.Graph;
8 import com.radarnetworks.kb.KnowledgeBase;
9 import com.radarnetworks.kb.KnowledgeBaseConnection;
10 import org.apache.log4j.Logger;
11
12 /**
13 * Plugin providing MOKB functionality in Desktop.
14 */
15 public class MOKBPluginImpl implements IMOKBPlugin {
16 private Logger log = Logger.getLogger(MOKBPluginImpl.class);
17 private IOAAKB oaaKB;
18 private IKnowledgeBaseFactory knowledgeBaseFactory;
19 private GraphConfig mokbGraphConfig;
20 private IGraphCreator graphCreator;
21 private String mokbGraphName;
22 private KnowledgeBaseConnection kbc;
23 private Graph jenaGraph;
24
25 public void startup() throws Exception {
26 log.info("Starting up MOKBPluginImpl plugin");
27 jenaGraph = graphCreator.createJDBMGraph(mokbGraphConfig);
28 knowledgeBaseFactory.addGraph(mokbGraphName, jenaGraph);
29 KnowledgeBase kb = (KnowledgeBase)knowledgeBaseFactory.createKnowledgeBase(mokbGraphName);
30 kbc = kb.getConnection();
31 }
32
33 public void shutdown () {
34 log.info("Shutting down MOKBPluginImpl plugin");
35 jenaGraph.close();
36 }
37
38 public IOAAKB getOaaKB() {
39 return oaaKB;
40 }
41
42 public void setOaaKB(IOAAKB oaaKB) {
43 this.oaaKB = oaaKB;
44 }
45
46 public IKnowledgeBaseFactory getKnowledgeBaseFactory() {
47 return knowledgeBaseFactory;
48 }
49
50 public void setKnowledgeBaseFactory(IKnowledgeBaseFactory knowledgeBaseFactory) {
51 this.knowledgeBaseFactory = knowledgeBaseFactory;
52 }
53
54 public GraphConfig getMokbGraphConfig() {
55 return mokbGraphConfig;
56 }
57
58 public void setMokbGraphConfig(GraphConfig mokbGraphConfig) {
59 this.mokbGraphConfig = mokbGraphConfig;
60 }
61
62 public String getMokbGraphName() {
63 return mokbGraphName;
64 }
65
66 public void setMokbGraphName(String mokbGraphName) {
67 this.mokbGraphName = mokbGraphName;
68 }
69
70 public IGraphCreator getGraphCreator() {
71 return graphCreator;
72 }
73
74 public void setGraphCreator(IGraphCreator graphCreator) {
75 this.graphCreator = graphCreator;
76 }
77 }Note how in the "startup" method, first a JDBM graph is created and then the graph is registered as a named graph. The name of the graph comes from the bean configuration. From the named graph the KnowledgeBase is created. From the KnowledgeBase is retrieved the KnowledgeBaseConnection.
See the documentation for IGraphCreator for an overview of how IRIS uses JDBM graphs for persistence.
The database graphs are stored in the application data directory. The class JDBMGraph is a custom JENA Graph used to store persisted triples. It is possible to create an instance of this class and read triples directly from disk. However, it is recommended instead to use the IRIS Knowledge Base APIs to retrieve ontology data.
IRIS also supports full text querying and maintains a Lucene Full Text Index for each graph (configured here). The next section documents the graph stack.
Database folders are subfolders named from the configuration. For example, the directory USER/.iris/jenadb contains the databse failes, and a subfolder for the full text index. The database files are specific to JDBM. An IRIS specific file named dbinfo.props exists that is used to tune the database. This file contains JDBM-specific parameters:
#Created Sat Dec 08 08:05:15 PST 2007 #Sat Dec 08 08:05:15 PST 2007 compressedStrings=true maxTransactionsInLog=10 nodeSerializer=DataStreamNodeSerializer.INSTANCE autoCommit=true commitInterval=1000 minCompressionSize=3072 compressedTriples=true recordManagerFactory=JDBMRecordManagerZipFactoryImpl.INSTANCE btreeSerializer=PreAllocatingLongArraySerializer.INSTANCE createdDate=1197129915375
It is not recommended to touch these files without being very familiar with JDBM internals. Note that there exists a utilitiy in IRIS to modify these parameters, and that utlitity will also automatically convert a JDBM database to a new set of parameters. This utlitity is used to tune the parameters and can be running by selecting the "IRIS Database Utlitity" startup option when running IRIS.
As documented in KB Separation IRIS implements ontology access using JENA Graphs. More specifically, IRIS uses a combination of JENA Graphs and custom Graphs to achieve a specific implementation which achieves IRIS reasoning goals and allows for good performance to be useable.
The graph stack is created by com.sri.iris.jena.factory.KnowledgeBaseFactory. It is posisble for IRIS components to create and register their own JENA Graphs with their own Graph stack and structure using the com.sri.iris.jena.factory.IKnowledgeBaseFactory APIs. The following is the default graph stack from top to bottom that gets created for each knowledge base:
The top-most graph interprets binding paths. This is done at the top so that property traversals through binding path segments will use all otehr graphs. This graph allows binding paths to be used in place of regular ontology properties. For example, it is possible to execute SPARQL queries using a binding path property and bind the subject and/or object.