Deploying to master from @ JanusGraph/janusgraph@e849077 🚀

Author: porunov

master/changelog/index.html

@@ -2000,21 +2000,28 @@
     Assets
   </a>
 
-    <nav class="md-nav" aria-label="Assets">
-      <ul class="md-nav__list">
+</li>
 
           <li class="md-nav__item">
   <a href="#upgrade-instructions" class="md-nav__link">
     Upgrade Instructions
   </a>
 
-</li>
+    <nav class="md-nav" aria-label="Upgrade Instructions">
+      <ul class="md-nav__list">
 
           <li class="md-nav__item">
   <a href="#inlining-vertex-properties-into-a-composite-index" class="md-nav__link">
     Inlining vertex properties into a Composite Index
   </a>
 
+</li>
+        
+          <li class="md-nav__item">
+  <a href="#berkeleyje-ability-to-overwrite-arbitrary-settings-applied-at-environmentconfig-creation" class="md-nav__link">
+    BerkeleyJE ability to overwrite arbitrary settings applied at EnvironmentConfig creation
+  </a>
+  
 </li>
 
       </ul>
@@ -3113,21 +3120,28 @@
     Assets
   </a>
 
-    <nav class="md-nav" aria-label="Assets">
-      <ul class="md-nav__list">
+</li>
 
           <li class="md-nav__item">
   <a href="#upgrade-instructions" class="md-nav__link">
     Upgrade Instructions
   </a>
 
-</li>
+    <nav class="md-nav" aria-label="Upgrade Instructions">
+      <ul class="md-nav__list">
 
           <li class="md-nav__item">
   <a href="#inlining-vertex-properties-into-a-composite-index" class="md-nav__link">
     Inlining vertex properties into a Composite Index
   </a>
 
+</li>
+        
+          <li class="md-nav__item">
+  <a href="#berkeleyje-ability-to-overwrite-arbitrary-settings-applied-at-environmentconfig-creation" class="md-nav__link">
+    BerkeleyJE ability to overwrite arbitrary settings applied at EnvironmentConfig creation
+  </a>
+  
 </li>
 
       </ul>
@@ -4360,7 +4374,7 @@ <h4 id="assets">Assets</h4>
 <li><a href="https://github.com/JanusGraph/janusgraph/releases/download/v1.1.0/janusgraph-1.1.0.zip">JanusGraph zip</a></li>
 <li><a href="https://github.com/JanusGraph/janusgraph/releases/download/v1.1.0/janusgraph-full-1.1.0.zip">JanusGraph zip with embedded Cassandra and ElasticSearch</a></li>
 </ul>
-<h5 id="upgrade-instructions">Upgrade Instructions</h5>
+<h4 id="upgrade-instructions">Upgrade Instructions</h4>
 <h5 id="inlining-vertex-properties-into-a-composite-index">Inlining vertex properties into a Composite Index</h5>
 <p>Inlining vertex properties into a Composite Index structure can offer significant performance and efficiency benefits.
 See <a href="../schema/index-management/index-performance/#inlining-vertex-properties-into-a-composite-index">documentation</a> on how to inline vertex properties into a composite index.</p>
@@ -4377,6 +4391,13 @@ <h5 id="inlining-vertex-properties-into-a-composite-index">Inlining vertex prope
    to revert to an older version of JanusGraph once this feature is used.</p>
 </li>
 </ol>
+<h5 id="berkeleyje-ability-to-overwrite-arbitrary-settings-applied-at-environmentconfig-creation">BerkeleyJE ability to overwrite arbitrary settings applied at <code>EnvironmentConfig</code> creation</h5>
+<p>The new namespace <code>storage.berkeleyje.ext</code> now allows to set custom configurations which were not directly exposed by 
+JanusGraph.
+The full list of possible setting is available inside the Java class <code>com.sleepycat.je.EnvironmentConfig</code>.
+All configurations values should be specified as <code>String</code> and be formated the same as specified in the official sleepycat
+<a href="https://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/EnvironmentConfig.html">documentation</a>.
+Example: <code>storage.berkeleyje.ext.je.lock.timeout=5000 ms</code></p>
 <h3 id="version-101-release-date">Version 1.0.1 (Release Date: ???)</h3>
 <div class="tabbed-set tabbed-alternate" data-tabs="2:2"><input checked="checked" id="__tabbed_2_1" name="__tabbed_2" type="radio" /><input id="__tabbed_2_2" name="__tabbed_2" type="radio" /><div class="tabbed-labels"><label for="__tabbed_2_1">Maven</label><label for="__tabbed_2_2">Gradle</label></div>
 <div class="tabbed-content">

master/configs/configuration-reference/index.html

@@ -942,6 +942,13 @@
     storage.berkeleyje
   </a>
 
+</li>
+        
+          <li class="md-nav__item">
+  <a href="#storageberkeleyjeext" class="md-nav__link">
+    storage.berkeleyje.ext
+  </a>
+  
 </li>
 
           <li class="md-nav__item">
@@ -2619,6 +2626,13 @@
     storage.berkeleyje
   </a>
 
+</li>
+        
+          <li class="md-nav__item">
+  <a href="#storageberkeleyjeext" class="md-nav__link">
+    storage.berkeleyje.ext
+  </a>
+  
 </li>
 
           <li class="md-nav__item">
@@ -4560,6 +4574,44 @@ <h3 id="storageberkeleyje">storage.berkeleyje</h3>
 </tr>
 </tbody>
 </table>
+<h3 id="storageberkeleyjeext">storage.berkeleyje.ext</h3>
+<p>Overrides for arbitrary settings applied at <code>EnvironmentConfig</code> creation.
+The full list of possible setting is available inside the Java class <code>com.sleepycat.je.EnvironmentConfig</code>. All configurations values should be specified as <code>String</code> and be formated the same as specified in the following <a href="https://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/EnvironmentConfig.html">documentation</a>.
+Notice, for compatibility reasons, it's allowed to use <code>-</code> character instead of <code>.</code> for config keys. All dashes will be replaced by dots when passing those keys to <code>EnvironmentConfig</code>.</p>
+<table>
+<thead>
+<tr>
+<th>Name</th>
+<th>Description</th>
+<th>Datatype</th>
+<th>Default Value</th>
+<th>Mutability</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>storage.berkeleyje.ext.je-lock-timeout</td>
+<td>Lock timeout configuration. <code>0</code> disabled lock timeout completely. To set lock timeout via this configuration it's required to use String formated time representation. For example: <code>500 ms</code>, <code>5 min</code>, etc.</td>
+<td></td>
+<td></td>
+<td></td>
+</tr>
+<tr>
+<td>See information about value constraints in the official <a href="https://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/EnvironmentConfig.html#LOCK_TIMEOUT">sleepycat documentation</a>.</td>
+<td></td>
+<td></td>
+<td></td>
+<td></td>
+</tr>
+<tr>
+<td>Notice, this option can be specified as <code>storage.berkeleyje.ext.je.lock.timeout</code> which will be treated the same as this configuration option.</td>
+<td>String</td>
+<td>(no default value)</td>
+<td>MASKABLE</td>
+<td></td>
+</tr>
+</tbody>
+</table>
 <h3 id="storagecql">storage.cql</h3>
 <p>CQL storage backend options</p>
 <table>

master/configs/janusgraph-cfg/index.html

@@ -2165,6 +2165,13 @@
     storage.berkeleyje
   </a>
 
+</li>
+      
+        <li class="md-nav__item">
+  <a href="#storageberkeleyjeext" class="md-nav__link">
+    storage.berkeleyje.ext
+  </a>
+  
 </li>
 
         <li class="md-nav__item">
@@ -4048,6 +4055,44 @@ <h3 id="storageberkeleyje">storage.berkeleyje</h3>
 </tr>
 </tbody>
 </table>
+<h3 id="storageberkeleyjeext">storage.berkeleyje.ext</h3>
+<p>Overrides for arbitrary settings applied at <code>EnvironmentConfig</code> creation.
+The full list of possible setting is available inside the Java class <code>com.sleepycat.je.EnvironmentConfig</code>. All configurations values should be specified as <code>String</code> and be formated the same as specified in the following <a href="https://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/EnvironmentConfig.html">documentation</a>.
+Notice, for compatibility reasons, it's allowed to use <code>-</code> character instead of <code>.</code> for config keys. All dashes will be replaced by dots when passing those keys to <code>EnvironmentConfig</code>.</p>
+<table>
+<thead>
+<tr>
+<th>Name</th>
+<th>Description</th>
+<th>Datatype</th>
+<th>Default Value</th>
+<th>Mutability</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>storage.berkeleyje.ext.je-lock-timeout</td>
+<td>Lock timeout configuration. <code>0</code> disabled lock timeout completely. To set lock timeout via this configuration it's required to use String formated time representation. For example: <code>500 ms</code>, <code>5 min</code>, etc.</td>
+<td></td>
+<td></td>
+<td></td>
+</tr>
+<tr>
+<td>See information about value constraints in the official <a href="https://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/EnvironmentConfig.html#LOCK_TIMEOUT">sleepycat documentation</a>.</td>
+<td></td>
+<td></td>
+<td></td>
+<td></td>
+</tr>
+<tr>
+<td>Notice, this option can be specified as <code>storage.berkeleyje.ext.je.lock.timeout</code> which will be treated the same as this configuration option.</td>
+<td>String</td>
+<td>(no default value)</td>
+<td>MASKABLE</td>
+<td></td>
+</tr>
+</tbody>
+</table>
 <h3 id="storagecql">storage.cql</h3>
 <p>CQL storage backend options</p>
 <table>

master/search/search_index.json

@@ -1405,6 +1405,20 @@
     Global Graph Operations
   </a>
 
+</li>
+      
+        <li class="md-nav__item">
+  <a href="#additional-berkeleydb-je-configuration-options" class="md-nav__link">
+    Additional BerkeleyDB JE configuration options
+  </a>
+  
+</li>
+      
+        <li class="md-nav__item">
+  <a href="#deadlock-troubleshooting" class="md-nav__link">
+    Deadlock troubleshooting
+  </a>
+  
 </li>
 
     </ul>
@@ -2024,6 +2038,20 @@
     Global Graph Operations
   </a>
 
+</li>
+      
+        <li class="md-nav__item">
+  <a href="#additional-berkeleydb-je-configuration-options" class="md-nav__link">
+    Additional BerkeleyDB JE configuration options
+  </a>
+  
+</li>
+      
+        <li class="md-nav__item">
+  <a href="#deadlock-troubleshooting" class="md-nav__link">
+    Deadlock troubleshooting
+  </a>
+  
 </li>
 
     </ul>
@@ -2118,6 +2146,43 @@ <h2 id="global-graph-operations">Global Graph Operations</h2>
 transactions enabled requires BerkeleyDB to acquire read locks on the
 data it is reading. When iterating over the entire graph, these read
 locks can easily require more memory than is available.</p>
+<h2 id="additional-berkeleydb-je-configuration-options">Additional BerkeleyDB JE configuration options</h2>
+<p>It's possible to set additional BerkeleyDB JE configuration which are not 
+directly exposed by JanusGraph by leveraging <code>storage.berkeleyje.ext</code> 
+namespace. </p>
+<p>JanusGraph iterates over all properties prefixed with
+<code>storage.berkeleyje.ext.</code>. It strips the prefix from each property key. 
+Any dash character (<code>-</code>) wil be replaced by dot character (<code>.</code>) in the 
+remainder of the stripped key. The final string will be interpreted as a parameter 
+ke for <code>com.sleepycat.je.EnvironmentConfig</code>. 
+Thus, both options <code>storage.berkeleyje.ext.je.lock.timeout</code> and 
+<code>storage.berkeleyje.ext.je-lock-timeout</code> will be treated 
+the same (as <code>storage.berkeleyje.ext.je.lock.timeout</code>). 
+The value associated with the key is not modified. 
+This allows embedding arbitrary settings in JanusGraph’s properties. Here’s an
+example configuration fragment that customizes three BerkeleyDB settings 
+using the <code>storage.berkeleyje.ext.</code> config mechanism:</p>
+<div class="highlight"><pre><span></span><code><span class="na">storage.backend</span><span class="o">=</span><span class="s">berkeleyje</span>
+<span class="na">storage.berkeleyje.ext.je.lock.timeout</span><span class="o">=</span><span class="s">5000 ms</span>
+<span class="na">storage.berkeleyje.ext.je.lock.deadlockDetect</span><span class="o">=</span><span class="s">false</span>
+<span class="na">storage.berkeleyje.ext.je.txn.timeout</span><span class="o">=</span><span class="s">5000 ms</span>
+<span class="na">storage.berkeleyje.ext.je.log.fileMax</span><span class="o">=</span><span class="s">100000000</span>
+</code></pre></div>
+<h2 id="deadlock-troubleshooting">Deadlock troubleshooting</h2>
+<p>In concurrent environment deadlocks are possible when using BerkeleyDB JE storage 
+backend. 
+It may be complicated to deal with deadlocks in use-cases when multiple threads are 
+modifying same vertices (including edges creation between affected vertices). 
+More insights on this topic can be found in the GitHub issue 
+<a href="https://github.com/JanusGraph/janusgraph/issues/1623">#1623</a>.</p>
+<p>Some users suggest the following configuration to deal with deadlocks:
+<div class="highlight"><pre><span></span><code><span class="na">storage.berkeleyje.isolation-level</span><span class="o">=</span><span class="s">READ_UNCOMMITTED</span>
+<span class="na">storage.berkeleyje.lock-mode</span><span class="o">=</span><span class="s">LockMode.READ_UNCOMMITTED</span>
+<span class="na">storage.berkeleyje.ext.je.lock.timeout</span><span class="o">=</span><span class="s">0</span>
+<span class="na">storage.lock.wait-time</span><span class="o">=</span><span class="s">5000</span>
+<span class="na">ids.authority.wait-time</span><span class="o">=</span><span class="s">2000</span>
+<span class="na">tx.max-commit-time</span><span class="o">=</span><span class="s">30000</span>
+</code></pre></div></p>