Data extraction client?

Thomas Sødring thomas.sodring at hioa.no
Tue Feb 14 15:49:14 CET 2017


On 02/14/2017 03:34 PM, Petter Reinholdtsen wrote:
> [Thomas Sødring]
>> Another thing that I've been thinking is that instead of having the
>> extraction tool internal to the core, we make it a stand alone system.
>> Probably a javaFX application that is capable of using the current
>> interface to create the extraction. This way it should be possible for
>> the code to be re-used by all Noark systems adhering to the standard.
>> The extraction part is going to be around version 0.6 I think. But a
>> fair bit of code exists already to make an extraction!
> I hope the API is complete enough to allow this, but given that I do not
> really know the details on what such data dump should contain, I do not
> know yet.
>From a first guess, there is enough in the API to support extraction
making!  Normally you might use a findAll method and process the
database in a particular fashion to make an extraction, but the
interface has no real findAll (avoid returning 300000 records in a
single call). Pagination is required so we'd process the core through
pages of results. It is possible!

I wrote some code  (php) to validate a Noark 5 extraction that might be
of interest:

https://github.com/KDRS-SA/noark5-validator

You can find two example extractions here:

https://github.com/KDRS-SA/noark5-validator/tree/master/src/resources/test-uttrekk

I think one is correct, while the other has some mistakes
>
> Such client would be very useful for the automated testing, to verify
> that the data stored look like it should after insertion.
>
> I thus want to include it in my test code, probably coded in python. :)
>
Good idea! We can develop it together if you want. You tend to be two
steps ahead so you will probably have it done first :) But I definitely
think that such a module is mandatory for a Noark 5 core.

 - Tom


More information about the nikita-noark mailing list