Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add skos lookup (#415) #229

Merged
merged 29 commits into from
Feb 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
b55bbbf
Add skos lookup (metafacture/metafacture-core#415)
dr0i Jun 13, 2022
2cba849
Fix usage without namespace (metafacture-core#415)
dr0i Sep 5, 2022
1819aff
Add follow URL redirections (metafacture-core#415)
dr0i Sep 6, 2022
5dea28e
Clean up formatting for SKOS lookup tests. (#229)
blackwinter Sep 8, 2022
7826062
Use camelCase (metafacture-core#415)
dr0i Sep 15, 2022
4b34178
Use withOption(...)
dr0i Sep 15, 2022
1d34730
Rename method (metafacture-core#415)
dr0i Sep 16, 2022
b11a77e
Set default value if not defined (metafacture-core#415)
dr0i Sep 16, 2022
234ab5a
Fix put_rdfmap (metafacture-core#415)
dr0i Sep 20, 2022
c3418a4
Remove static defaultValue (metafacture-core#415)
dr0i Sep 20, 2022
94ae82a
Extract lookup into FixFunction (metafacture-core#415)
dr0i Sep 20, 2022
62da8a0
Add slf4j-log4j12 dependency
dr0i Sep 22, 2022
29d1360
Fix order; rename variable; remove empty line (metafacture-core#415)
dr0i Sep 23, 2022
5d70f14
Fix typos (metafacture-core#415)
dr0i Sep 23, 2022
9dd9762
Add integration tests for Skos-Lookup
TobiasNx Sep 29, 2022
b166fd2
Enable further use cases
dr0i Oct 7, 2022
219f2bf
Fix test lookupRdfDefinedPropertyToProperty
TobiasNx Oct 27, 2022
d94728b
Add optional parameter "select"
dr0i Nov 4, 2022
1d0e542
Enable further test and fix Fix (metafacture-core#415)
dr0i Nov 18, 2022
b49445d
Rename "target_language" to "select_language"
dr0i Nov 22, 2022
9251ca3
Rename integration tests for rdf lookup
TobiasNx Nov 25, 2022
c81ae7d
Tweak rdf lookup (metafacture-core#415)
dr0i Dec 13, 2022
253e25c
Add wiremock to test RDF url (metafacture-core#415)
dr0i Dec 27, 2022
eaf08c2
Drop unused methods. (94ae82a)
blackwinter Feb 9, 2023
0987eb0
Drop special treatment of RDF maps in `lookup()`. (#229)
blackwinter Feb 9, 2023
dc9e63d
Drop special treatment of map name in `put_rdfmap()`. (#229)
blackwinter Feb 9, 2023
897c21d
Unify URI identification logic for RDF maps. (#229)
blackwinter Feb 9, 2023
7da6541
A little cleanup. (#229)
blackwinter Feb 9, 2023
3c09175
Fix logging setup. (62da8a0)
blackwinter Feb 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,17 @@ put_map("<mapName>",
)
```

##### `put_rdfmap`

Defines an external RDF map for lookup from a file or an HTTP(S) resource.
As the RDF map is reducing RDF triples to a key/value map it is mandatory to set the target.
The targeted RDF property can optionally be bound by an RDF language tag.

```perl
put_rdfmap("<rdfResource>", "<rdfMapName>", target: "<rdfProperty>")
put_rdfmap("<rdfResource>", "<rdfMapName>", target: "<rdfProperty>", select_language: "<rdfLanguageTag>")
```

##### `put_var`

Defines a single global variable that can be referenced with `$[<variableName>]`.
Expand Down Expand Up @@ -553,7 +564,7 @@ join_field("<sourceField>", "<separator>")

##### `lookup`

Looks up matching values in a map and replaces the field value with this match. [External files](#put_filemap) as well as [internal maps](#put_map) can be used.
Looks up matching values in a map and replaces the field value with this match. [External files](#put_filemap), [internal maps](#put_map) as well as [RDF resources](#put_rdfmap) can be used.

Parameters:

Expand Down Expand Up @@ -599,6 +610,10 @@ lookup("path.to.field", "path/to/file", sep_char: ";")
put_filemap("path/to/file", "file-map", sep_char: ";")
lookup("path.to.field", "file-map")

# RDF map (explicit)
put_rdfmap("path/to/file", "rdf-map", target: "<rdfProperty>")
lookup("path.to.field", "rdf-map")

# with default value
lookup("path.to.field", "map-name", __default: "NA")

Expand Down
6 changes: 4 additions & 2 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@ subprojects {
'ace': '1.3.3',
'antlr': '3.2',
'equalsverifier': '3.8.2',
'guava': '29.0-jre',
'jackson': '2.13.3',
'jena': '3.17.0',
'jetty': '9.4.14.v20181114',
'jquery': '3.3.1-1',
'junit_jupiter': '5.8.2',
Expand All @@ -45,8 +47,8 @@ subprojects {
'mockito': '2.27.0',
'requirejs': '2.3.6',
'slf4j': '1.7.21',
'xtext': '2.26.0',
'guava': '29.0-jre'
'wiremock': '2.33.2',
'xtext': '2.26.0'
]
}

Expand Down
11 changes: 9 additions & 2 deletions metafix/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,18 @@ dependencies {
implementation "com.fasterxml.jackson.core:jackson-core:${versions.jackson}"
implementation "com.fasterxml.jackson.core:jackson-databind:${versions.jackson}"
implementation "com.google.guava:guava:${versions.guava}"
implementation "org.apache.jena:jena-arq:${versions.jena}"
implementation "org.apache.jena:jena-core:${versions.jena}"
implementation "org.eclipse.emf:org.eclipse.emf.ecore:${versions.xtext}" // Workaround for hbz/lobid-resources#1462
implementation "org.eclipse.xtext:org.eclipse.xtext.xbase:${versions.xtext}"
implementation "org.eclipse.xtext:org.eclipse.xtext:${versions.xtext}"
implementation "org.slf4j:slf4j-api:${versions.slf4j}"

testImplementation "org.junit.jupiter:junit-jupiter-api:${versions.junit_jupiter}"
testImplementation "org.junit.platform:junit-platform-launcher:${versions.junit_platform}"
testImplementation "com.github.tomakehurst:wiremock-jre8:${versions.wiremock}"
testImplementation "org.eclipse.xtext:org.eclipse.xtext.testing:${versions.xtext}"
testImplementation "org.eclipse.xtext:org.eclipse.xtext.xbase.testing:${versions.xtext}"
testImplementation "org.junit.jupiter:junit-jupiter-api:${versions.junit_jupiter}"
testImplementation "org.junit.platform:junit-platform-launcher:${versions.junit_platform}"

testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:${versions.junit_jupiter}"

Expand All @@ -43,6 +46,10 @@ dependencies {
configurations {
mwe2 {
extendsFrom implementation

dependencies {
implementation "org.slf4j:slf4j-simple:${versions.slf4j}"
}
}
}

Expand Down
17 changes: 17 additions & 0 deletions metafix/src/main/java/org/metafacture/metafix/FixMethod.java
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
package org.metafacture.metafix;

import org.metafacture.metafix.api.FixFunction;
import org.metafacture.metafix.maps.RdfMap;
import org.metafacture.metamorph.api.Maps;
import org.metafacture.metamorph.functions.ISBN;
import org.metafacture.metamorph.functions.Timestamp;
Expand Down Expand Up @@ -92,6 +93,22 @@ public void apply(final Metafix metafix, final Record record, final List<String>
metafix.putMap(params.get(0), options);
}
},
put_rdfmap {
@Override
public void apply(final Metafix metafix, final Record record, final List<String> params, final Map<String, String> options) {
final String fileName = params.get(0);
final RdfMap rdfMap = new RdfMap();

rdfMap.setResource(fileName, metafix::resolvePath);

withOption(options, RdfMap.TARGET, rdfMap::setTarget);
withOption(options, RdfMap.TARGET_LANGUAGE, rdfMap::setTargetLanguage);
withOption(options, RdfMap.SELECT, rdfMap::setSelect);
withOption(options, Maps.DEFAULT_MAP_KEY, rdfMap::setDefault);

metafix.putMap(params.size() > 1 ? params.get(1) : fileName, rdfMap);
}
},
put_var {
@Override
public void apply(final Metafix metafix, final Record record, final List<String> params, final Map<String, String> options) {
Expand Down
Loading