<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi,</p>
<p><br>
</p>
<p>Good stuff! Let's be honest, that's not good reading. </p>
<p><br>
</p>
<p>The amount of data being stored is very small so 41 KiB per entry is strange. If you are using H2 (in memory db) then I understand that there is a </p>
<p>cost as indexes etc are being created. But we should also test on top of mysql/postgres. I'm not sure what you have persisted the data to.</p>
<p><br>
</p>
<p>There are 3 things that should be considered before benchmarking.</p>
<p><br>
</p>
<p></p>
<ol style="margin-bottom:0px; margin-top:0px">
<li>Database</li><li>In demo mode, we run in log DEBUG</li><li>Hibernate is printing all SQL commands</li></ol>
<p></p>
<p><br>
</p>
<p>To change 2.</p>
<p><a href="https://github.com/HiOA-ABI/nikita-noark5-core/blob/master/core-webapp/src/main/resources/application-demo.yml#L13" class="x_OWAAutoLink" id="LPlnk124814">https://github.com/HiOA-ABI/nikita-noark5-core/blob/master/core-webapp/src/main/resources/application-demo.yml#L13</a><br>
<br>
</p>
<p>should be set INFO or WARN</p>
<p><br>
</p>
<p>To change 3.</p>
<p><a href="https://github.com/HiOA-ABI/nikita-noark5-core/blob/master/core-webapp/src/main/resources/application-demo.yml#L38" class="x_OWAAutoLink" id="LPlnk156044">https://github.com/HiOA-ABI/nikita-noark5-core/blob/master/core-webapp/src/main/resources/application-demo.yml#L38</a><br>
<br>
</p>
<p>Set this to false.</p>
<p><br>
</p>
<p>This is not going cause a major increase in processing time, but it's something.</p>
<p><br>
</p>
<p>When it comes to 1. <span style="font-size:12pt">I believe there are a few indexes in place so that might explain the memory usage. We just need to run on mysql to check.</span></p>
<p><br>
</p>
<p>Maybe I should add a description to get mysql integration working, it should work out of the box but it's been so long since I looked at the code.</p>
<p><br>
</p>
<p> - Tom</p>
<p><span style="font-size:12pt"><br>
</span></p>
<p><span style="font-size:12pt"><br>
</span></p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> nikita-noark-bounces@nuug.no <nikita-noark-bounces@nuug.no> on behalf of Petter Reinholdtsen <pere@hungry.com><br>
<b>Sent:</b> Saturday, May 13, 2017 6:04:39 PM<br>
<b>To:</b> nikita-noark@nuug.no<br>
<b>Subject:</b> Benchmarking the insertion of 8333 post journal entries</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText"><br>
Just for fun, I did a small benchmark of the Nikita Noark 5 Core API<br>
running on my laptop (machine is slighly short on memory), to see of<br>
much time the sqlite2noark5core script would use to insert 8333 into the<br>
database via the API.<br>
<br>
This took 64 minutes and 24 seconds. According to top, the java process<br>
started with sizes virt 5580736, res 819272 and shr 23052. When the job<br>
was done, the java process had sizes virt 5604600 res 1.109g and shr 0.<br>
<br>
This mean the script was able to insert around 2.2 entries per second,<br>
and the java process resident size grew 343599 KiB, ie 41 KiB per entry.<br>
<br>
For each entry the script create a 'saksmappe' entry, a 'journalpost'<br>
entry, zero or more 'korrespondansepart' entries and finally one<br>
'dokumentbeskrivelse' entry. At the start it creates one 'arkiv' and<br>
'arkivdel' entry.<br>
<br>
Thought it best to mention the numbers here to have some kind of<br>
baseline when we start to look at profiling the API to speed it up. :)<br>
<br>
-- <br>
Happy hacking<br>
Petter Reinholtsen<br>
_______________________________________________<br>
nikita-noark mailing list<br>
nikita-noark@nuug.no<br>
<a href="https://lists.nuug.no/mailman/listinfo/nikita-noark">https://lists.nuug.no/mailman/listinfo/nikita-noark</a><br>
</div>
</span></font>
</body>
</html>