Multisensor Validation Log

It would be good if the crawler decodes HTML entities before storing in dc:subject (possibly also dc:title, dc:description). Eg:

dc:subject
  "Europäische Union, ISIN_FR0003500008, Brexit, Erholungskurs, Europa, Paris, Großbritannien" ,
  "Economy, Business & Finance" ;

Victor: both of the following fields extracted by the crawler are mapped to dc:subject:

• category: eg "Economy, Business & Finance"
• keywords: eg "Europaische Union, ISIN_FR0003500008, Brexit, Erholungskurs, Europa, Paris, Grossbritannien"

Should we separate them in different properties?

Vladimir: IPTC is the Global Standards Body of the News Media industry. IPTC Media Topics is a list of 1100 Media Topics developed as an extension from earlier IPTC Subject Codes. You can explore interactively and will see that Multisensor categories are similar to these topics/subjects; so we'll continue to map them to dc:subject.

On the other hand, Multisensor keywords are free keywords that describe a lot more specific things. We'll map them to schema:keywords, defined as "Keywords or tags used to describe the content. Multiple entries in a keywords list are typically delimited by commas":

dc:subject "Economy, Business & Finance";
schema:keywords "Europaische Union, ISIN_FR0003500008, Brexit, Erholungskurs, Europa, Paris, Grossbritannien";

It would be nice to:

• Split keywords on ", " and and emit as separate values
  • (don't split categories, since the 3 words really represent one category)
• Map our categories to IPTC Media Topics. This is quite harder

<2016-07-20 Wed> Almost done, only the datatype is missing.

Victor: introduce the date when the article have been processed in RDF. In order to keep track of which and when the “curated selected data” have been processed, and match them with the current version of the CEP service.

Vladimir: we can use dct:issued for this purpose:

ms-content:b3f35
  dc:date    "2016-06-20T18:45:07.000+02:00"^^xsd:dateTime ; # date crawled
  dct:issued "2016-06-30T12:34:56.000+02:00"^^xsd:dateTime ; # date processed by pipeline and ingested to GDB

Boyan will add this second timestamp in the POST method.

Status:

• <2016-07-18> DONE: initial implementation
• <2016-07-20> DONE: make dqv:value nominal (eg ms:accuracy-low) instead of numeric (eg 1)
• <2016-07-26> DONE: use QualityAnnotation instead of QualityMeasurement

Victor: the field "c_quality" is sent now. Values can be:

• 0 = no quality assigned
• 1 = high quality
• 2 = medium quality
• 3 = low quality
• 5 = curated

Vladimir:

• Instead of a numeric scale (which is not monotonically increasing), let's use a nominal (symbolic) scale.
• Instead of 0, we should omit the statement
• There is no value 4
• About value 5: do we have metadata who & when curated it? Should we record in RDF something more than the number? The selected quality ontology (DQV, see below) allows to record a lot of detail: who, when, according to what methodology…
  • Leszek (skype:letschke): we do not provide any additional meta data.

Vladimir: I searched for a quality property on LOV, couldn't find anything really appropriate:

• http://www.w3.org/ns/dcat#dataQuality: this is about datasets, but is deprecated: "This should not be used to describe the data collection characteristics, other more specialized statistical properties can be used instead". But I don't see such statistical properties
• http://def.seegrid.csiro.au/isotc211/iso19115/2003/metadata#dataQualityInfo: this is about ISO 19115 "Geographic information — metadata" http://def.seegrid.csiro.au/isotc211/iso19115/2003/dataquality is a whole separate module on Quality
• http://purl.oclc.org/NET/ssnx/ssn#qualityOfObservation: this is about Semantic Sensor Networks. It makes reference to resultQuality in ISO 19156 "Geographic information — Observations and measurements"

Finally from a link at Europeana DQC, I found the W3C Data Quality Vocabulary dqv:. We'll use that vocabulary, and the Linked Data Quality Dimensions ldqd: by Zaveri.

select * {?x a foaf:Document} 
# 112k SIMMOs
select * {?x a foaf:Document; dc:creator ?y}
# 10.5k authors, only 9.4%

• Can we get more authors?

Discussion <2016-07-08 Fri> MULTISENSO-186:

• Andrey: Interest in the "genre", “author” feature if available (not always retrievable by the context extraction service)
• Ioannis: The genre and author information can only be extracted when they are available in the HTML content of the scrapped page, otherwise we cannot infer it. To this end, there is not much we can do. It was quite obvious on the planning stage that not all articles have mentioned fields in HTML tags, and it could be foreseen, maybe with additional parsing methods since not always html tags do have this information, but normally all news articles have for example Author info in the body of the article. So this part can be closed, as "it" will not going to happen.

Vladimir & Ioannis: by Genre, do you mean dc:type? We assume so below.

select * {?x a foaf:Document; dc:type ?y} 
# 20k, that's 17.9%

• Q: Can we get more genres?
• A: Same comments as in the previous section apply.

Distribution of Genre:

select ?y (count(*) as ?c)
{?x a foaf:Document; dc:type ?y}
group by ?y order by desc(?c)

<2016-07-08 Fri>

• Vladimir: it would be nice to normalize some values, and reduce this from 20 to say 10 rows
• Ioannis: the code to extract "type" was written over a year ago, so this would not be so simple
• Vladimir and Ioannis: the first 5 types catch 90%, so it's only the "long tail" would need normalization… This is not so important

Advanced Context Extraction adds new Text Characteristics properties (ms:technicality, ms:fluency, ms:richness) to the context (property definitions omitted for brevity).

Example (this particular text is fluent, but is neither rich nor technical, so we set values 5, 1, 1 respectively):

@base <http://data.multisensorproject.eu/content/9e9c304>.

<#char=0,2000> a nif:Context;
  nif:isString "This is the whole text of the SIMMO.\n It should continue for 2000 chars but I'll stop here"@en;
  ms:fluency      5.0;
  ms:richness     1.0;
  ms:technicality 1.0.

<2016-06-23 Thu>: checked 0e6b24-CONTEXT_EXTRACTION-22-6-2016.ttl and Text Characteristics (technicality, fluency, richness):

• currently use http://data.multisensor.eu/ontology#
• but the correct prefix is http://data.multisensorproject.eu/ontology#

Victor <2016-06-30 Thu>: updated the prefix

I've added http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-annotation# to ./img/prefixes.ttl (committed <2016-06-30 Thu>) Please refresh from ./img/prefixes.ttl, so new validation files use this prefix.

Gerard: what exactly are we suppposed to do with the prefixes file.

• Once loaded in the repo (Boyan's job), we can make queries without mentioning the prefixes.

Should we load it into Sesame somehow so that triples are generated with prefixes? If so, could you give us some code showing how to do it?

• I think the validation files use prefixes, because prefixes.ttl is prepended, then passed through RIOT. I think Victor made that script. The other prefixes are there, so it's just a matter of refreshing

The EL service emits Babelnet entity labels in up to 4 languages, eg

bn:s00088614v  skos:prefLabel  "zu_befriedigen"@de , "satisfacer"@es , "satisfaire"@fr , "задоволи"@bg .
bn:s00014609n  skos:prefLabel  "Kuchen"@de , "Pastel_(gastronomía)"@es , "Gâteau"@fr , "Торта"@bg .
bn:s01718102n  skos:prefLabel  "I_do_not_want_what_I_haven't_got"@es , "I_Don't_Want_What_I_Haven't_Got"@en , "I_Do_Not_Want_What_I_Haven't_Got"@fr .
bn:s02229586n  skos:prefLabel  "UHC_Hamburg"@en , "Uhlenhorster_HC"@fr .

For reasons unknown, Babelnet uses underscores (eg see UHC_Hamburg_n_EN). I think we should convert the underscores to spaces to make the label more natural.

• Can be fixed in the repo with a query like this
  • However this doesn't find Babelnet broader entities imported by ONTO. We have a list of all Babelnet entities, maybe better to use this somehow

delete {?x skos:prefLabel ?lab}
insert {?x skos:prefLabel ?lab1}
where {
    ?x skos:prefLabel ?lab
    filter exists {[its:taIdentRef ?x; nif-ann:taIdentProv <http://babelfy.org/>]}
    filter(regex(?lab,"_"))
    bind(replace(?lab,"_"," ") as ?lab1)
}

• UPF code that brings in new Babelnet enrichments should be fixed too. Gerard: DONE <2016-07-20 Wed> Entitly Linking: Translations obtained from BabelNet are now issued as literals of skos:prefLabel triples without underscores.
• UPF code that creates nif:lemma should be fixed, eg this node has lemma "basic_data" http://multisensor.ontotext.com/resource/ms-content/00a17bdb91543c45349f42378caeecd434c1a8f4#char=281,291 Gerard: DONE <2016-07-20 Wed>. Dependency parsing: lemmas are also emitted as literals of nif:lemma triples without underscores
• IMPORTANT <2016-08-12 Fri> Now we have two labels per BN concept per language. Must remove the superfluous ones. Eg:
  • bn:s03113558n: "Royal Ordnance Factories F.C."@en and "Royal_Ordnance_Factories_F.C."@en
  • bn:s00124949n: "Prefijo del Segmento de Programa"@es and "Prefijo_del_Segmento_de_Programa"@es

Should we also remove disambiguations, which are trailing parenthesized parts, eg "Pastel_(gastronomía)" -> "Pastel"? Since these labels are not used for NLP tasks, and the disambiguations are very useful for understanding what the entity is, I vote to leave them.

In the example above, taClassRef and taIdentRef were removed. This makes the other two props nif-ann:taIdentConf and nif-ann:taIdentProv useless. Remove them too.

Entity Alignment also seems to remove the dependency links, eg:

<#char=1444,1452> nif:dependency          <#char=1453,1461>
<#char=1444,1452> upf-deep:deepDependency <#char=1453,1461>

However, this can make the dependency and FrameNet graphs disconnected. So leave the dependencies alone.

I shortened the excerpt from alignment.log above to improve readability: substituted the defined prefixes, and used the SIMMO URL as base (i.e. used relative URLs starting with hash) It would be very useful if alignment.log uses the same shortenings to improve readability.

This is a completely cosmetic issue, we can cancel it.

I've been saying all along to skip nif:anchorOf so as not to create too many literals. But with the number of SIMMOs loaded, it has not been too taxing for GDB. nif:anchorOf has been instrumental in debugging, eg of the UTF-8 and offset mismatch issues.

nif:literalAnnotation and nif:lemma provide sufficient info about the phrase, so maybe we don't need nif:anchorOf. We could cancel this issue.

• Gerard: If they can be sustained by GraphDB, I vote in favor of keeping them as they help a lot when debugging.
• Vladimir: so decided: if the Entity Lookup makes a new node, add nif:anchorOf to it.

Some nodes are missing nif:anchorOf, eg see http://multisensor.ontotext.com/resource/ms-content/00a17bdb91543c45349f42378caeecd434c1a8f4#char=281,291. This has nif:lemma "basic_data" but not nif:anchorOf.

Gerard <2016-07-20 Wed> NIFUtils: new annotations created by services using this library will now emit anchors. This affects EL mostly

In the above example, nif-ann:taIdentConf is 0. In many other examples it's a good number, eg see below. Is 0 some sort of bug, or does Babelfy actually return 0 confidence for some concepts?

• Gerard: I think it's an error, I'll get back to you as soon as as I've figured out what's causing it.
• Vladimir <2016-09-05 Mon> Still occurs, eg

  <3c361-de#char=8190,8198>
        a                      nif:Phrase , nif:Word ;
        nif-ann:taIdentConf    "0.0"^^xsd:double ;
        nif-ann:taIdentProv    <http://babelfy.org/> ;
        nif:anchorOf           "Zugleich" ;
  

bf6fe4-CONCEPT_EXTRACTION-16-6-2016.ttl

@base <http://data.multisensorproject.eu/content/bf6fe48b8d88c1d11d5086863f4c3ad26286bda9>.

<#char=1814,1822>
        a                        nif:Word ;
        nif-anno:taIdentConf     "0.7619547411890493"^^xsd:double ;
        nif-anno:taIdentProv     <http://babelfy.org/> ;
        nif:anchorOf             "pastries" ;
        nif:beginIndex           "1814"^^xsd:nonNegativeInteger ;
        nif:dependency           <#char=1806,1812> ;
        nif:endIndex             "1822"^^xsd:nonNegativeInteger ;
        nif:lemma                "pastry" ;
        nif:literalAnnotation
          "deep=spos=NN" , 
          "rel==member=A2|dpos=NN|end_string=1822|start_string=1814|id0=29|word=pastry|number=PL|connect_check=OK|fn=Food" , 
          "surf=spos=NN" ;
        nif:oliaLink             upf-deep:COORD , penn:NNS , <#char=1814,1822_fe> , upf-dep-syn:COORD ;
        nif:referenceContext     <#char=0,12793> ;
        upf-deep:deepDependency  <#char=1806,1812> ;
        its:taClassRef           ms:GenericConcept ;
        its:taIdentRef           bn:s00060957n .

• Gerard (about the last example): A 'generic' concept produced by Babelfy. But annotations of concepts produced by the concept extraction service should contain triples pointing to ms:SpecificConcept.
• Vladimir: also seems to me that concepts like "open source" and "pastry" are ms:SpecificConcept.
• Gerard: problems regarding the quality of the annotations shouldn't be part of the RDF validation.
• Vladimir: agree, but this log is for the project as a whole, not just syntactic validity. (Which doesn't mean I'm determining priorities!)
• Gerard: we'll be releasing updates to the concept extraction service, so expect improvements into what is considered a specific concept.
• How is this used in the UI? Gerard thinks that only Specific concepts are (or should be) shown in the SIMMO's tag cloud

Gerard <2016-07-20 Wed> This should become a non issue after recent changes are incorporated to the concept service

Gerard wrote some of the Summarization queries are slow. Please mark which ones need optimization, and provide $graph for testing.

• Used the standard notation $param to indicate an input parameter, rather than __PARAM__
• Moved FILTER inside GRAPH, and a few more minor changes
• The problem was that the prop path p1?/p2 is slow, since p1? connects any node to itself. Replaced with p1/p2|p2, which is fast

<2016-06-28 Tue> checked CAP:_Specification_of_the_service. It proposes the following model:

<http://data.multisensorproject.eu/content/53a0938bc4770c6ba0e7d7b9ca88a637f9e9c304#similarArticles>
  a oa:Annotation ;
  oa:hasTarget <http://data.multisensorproject.eu/content/53a0938bc4770c6ba0e7d7b9ca88a637f9e9c304> ;
  oa:hasBody        
    <http://data.multisensorproject.eu/content/53a0938bc4770c6ba0e7d7b9ca88a637f9e9c304#similarArticle-1> ,
    <http://data.multisensorproject.eu/content/53a0938bc4770c6ba0e7d7b9ca88a637f9e9c304#similarArticle-2> ;
  oa:motivatedBy oa:tagging ;
  oa:annotatedBy <http://data.multisensorproject.eu/agent/CAPAgent> ;
  oa:annotatedAt "2016-01-11T12:00:00"^^xsd:dateTime .

<http://data.multisensorproject.eu/content/53a0938bc4770c6ba0e7d7b9ca88a637f9e9c304#similarArticle-1>
  a oa:SemanticTag ;
  skos:related <http://data.multisensorproject.eu/content/ca34bb35770bfa55434a0689d64e1e6a60611047> ;
  fise:confidence 0.862 .

<http://data.multisensorproject.eu/content/53a0938bc4770c6ba0e7d7b9ca88a637f9e9c304#similarArticle-2>
  a oa:SemanticTag ;
  skos:related <http://data.multisensorproject.eu/content/57e07befbda355c2eca2ee521926071ee9f5c719> ;
  fise:confidence 0.795 .

<http://data.multisensorproject.eu/agent/CAPAgent>
  a prov:SoftwareAgent ;
  foaf:name "Content Alignment Pipeline v1.0" .

Each annotation is symmetric, so it's written twice: in the SIMMO graphs of each of the two SIMMOs. This complicates data management, because both of these graphs need to be updated.

After consultation with Babis, we decided to change the representation as follows:

• Write annotations in their own graph http://data.multisensorproject.eu/CAP, outside of any SIMMO graph. The CAP service will be called periodically, search globally in the SIMMO DB, and overwrite the similarity graph.
• Write one annotation per pair
• Use custom oa:motivatedBy: ms:linking-similar vs ms:linking-contradictory to express similarity vs contradiction

In the previous example, assume that the first related article is similar but the second is contradictory. We restructure it as follows, where similarity/123 and similarity/124 are GUIDs or some other way to generate unique URLs. Please note that the representation is completely symmetric regarding the two SIMMOs being linked, so there's no need to repeat for the other SIMMO.

graph <http://data.multisensorproject.eu/CAP> {
  <http://data.multisensorproject.eu/CAP/123> a oa:Annotation;
    oa:hasBody        
      <http://data.multisensorproject.eu/content/53a0938bc4770c6ba0e7d7b9ca88a637f9e9c304>,
      <http://data.multisensorproject.eu/content/ca34bb35770bfa55434a0689d64e1e6a60611047>;
    fise:confidence 0.862;
    oa:motivatedBy ms:linking-similar;
    oa:annotatedBy <http://data.multisensorproject.eu/agent/CAP>;
    oa:annotatedAt "2016-01-11T12:00:00"^^xsd:dateTime .

  <http://data.multisensorproject.eu/CAP/124> a oa:Annotation;
    oa:hasBody        
      <http://data.multisensorproject.eu/content/53a0938bc4770c6ba0e7d7b9ca88a637f9e9c304>,
      <http://data.multisensorproject.eu/content/57e07befbda355c2eca2ee521926071ee9f5c719>;
    fise:confidence 0.795;
    oa:motivatedBy ms:linking-contradictory;
    oa:annotatedBy <http://data.multisensorproject.eu/agent/CAP>;
    oa:annotatedAt "2016-01-12T12:00:00"^^xsd:dateTime .
}

In an example sent by Babis, I see fise:confidence=1.6439653807554948. But confidence is the probability that something is true, so it should be <=1. Guess this is some other sort of score, and maybe it's better to use our own property?

Decided with Babis to use a custom property ms:score (see next).

The following will be in ./img/ontology.ttl, so they don't need to be repeated by CAP for every similarity link.

<http://data.multisensorproject.eu/agent/CAP> a prov:SoftwareAgent;
  foaf:name "Content Alignment Pipeline v1.0".

ms:linking-similar a owl:NamedIndividual, oa:Motivation;
  skos:inScheme oa:motivationScheme;
  skos:broader oa:linking;
  skos:prefLabel "linking-similar"@en;
  rdfs:comment "Motivation that represents a symmetric link between two *similar* articles"@en;
  rdfs:isDefinedBy ms: .

ms:linking-contradictory a owl:NamedIndividual, oa:Motivation;
  skos:inScheme oa:motivationScheme;
  skos:broader oa:linking;
  skos:prefLabel "linking-contradictory"@en;
  rdfs:comment "Motivation that represents a symmetric link between two *contradictory* articles"@en;
  rdfs:isDefinedBy ms: .

ms:score a owl:DatatypeProperty;
  rdfs:domain oa:Annotation;
  rdfs:range  xsd:decimal;
  rdfs:label "score"@en;
  rdfs:comment "Strength of an Annotation, eg the link between two entities"@en;
  rdfs:isDefinedBy ms: .

Given a $simmo, find similar or contradictory articles, and their similarity/contradiction scores.

select ?article ?motivation ?score {
  [a oa:Annotation;
   oa:annotatedBy <http://data.multisensorproject.eu/agent/CAP>;
   oa:hasBody $simmo, ?article;
   ms:score ?score;
   oa:motivatedBy ?motivation
  ]
  filter ($simmo != ?article)
}

The gdoc maybe has 2 queries related to CAP. Not sure I'm looking at the right section. Maybe we should just delete them.

• 2.8 "Retrieve the concepts in the SIMMO (Select)": wrote something simple
• 2.9 "Retrieve the concepts in the SIMMO (Construct)": don't know what is needed

The Entity Linking Service can find BN concepts in any language, because BN concepts have multilingual labels. This is unrelated to whether the SIMMO is translated to another language or not. It's up to the UI to filter BN labels and show only labels in the selected language.

However, not all BN concepts have labels in all MS languages. Therefore the UI should implement some language fall-back logic:

• if no label in the selected language is available, show the EN label
• TODO Boyan: check whether all BN concepts have EN labels.

Gerard, as we saw above, Babelfy recognizes "batería de iones de litio" but bn:s01289274n does not have ES label (has EN, FR, DE, BG). This concept has ES representation on the BN site. Do you need to add Spanish labels?

• Gerard <2016-09-01>: The EL service is using the 2.5.1 release of BabelNet to fetch the translations, as these cannot be retrieved using Babelfy's API, so it is possible that some synset senses have translations in the latest version online (suppose it matches 3.7) which aren't found in 2.5.1.
• Vladimir: When we got complete BN data (for the concepts found by you plus their ancestors), we used BN online.

So http://babelnet.org/rdf/s01157392n should include all needed labels. (But right now it is down: "Error parsing configuration file file://c:/home/debian/storage/apache-tomcat-7.0.65/webapps/rdf/WEB-INF/config.ttl: Error making the query, see cause for details")