Skip to content

Commit 394e245

Browse files
authored
[ZEPPELIN-6400] Remove ZeppelinConfiguration dependency from zeppelin-interpreter module
## Summary - **Move `ZeppelinConfiguration` from `zeppelin-interpreter` to `zeppelin-zengine`** so it is no longer included in the shaded interpreter JAR. This prevents the Maven shade plugin from corrupting config string literals, which caused classpath-order-dependent configuration loading failures. - **Replace `ZeppelinConfiguration` usage in `zeppelin-interpreter` with `Properties`-based configuration** across `InterpreterLauncher`, `LifecycleManager`, `RecoveryStorage`, `DependencyResolver`, and all launcher plugins (Docker, K8s, YARN, Flink). - **Update callers** in `zeppelin-zengine`, `zeppelin-server`, `flink`, and `markdown` interpreter modules. - **Fix `TimeoutLifecycleManager` to parse time unit suffixes** (e.g., `"10s"`, `"1000ms"`) by adding `parseTimeValue()`. Previously, `Long.parseLong("10s")` threw `NumberFormatException`, causing the `TimeoutLifecycleManagerTest.testTimeout_2` to enter an infinite loop and hang CI for 6 hours. - **Fix flaky `PersonalizeActionsIT.testGraphAction` Selenium test** by using `clickAndWait()` instead of `clickableWait().click()`, allowing the UI to update before assertion. ## Motivation `ZeppelinConfiguration` in `zeppelin-interpreter` gets processed by the Maven shade plugin, which corrupts string literals (e.g., `org.apache.zeppelin` → `unshaded.org.apache.zeppelin`). This causes config keys to mismatch at runtime depending on classpath ordering. Moving it to `zeppelin-zengine` (which is not shaded) permanently eliminates this class of bugs. As discussed by the community: *"ZeppelinConfiguration belongs to the Zeppelin server, and the Zeppelin interpreter should really only work on a HashMap with ConfigKey and ConfigValue."* ## Changes | Area | Change | |------|--------| | `zeppelin-interpreter` (core) | Remove `ZeppelinConfiguration` imports; use `Properties` for config | | `InterpreterLauncher` | `ZeppelinConfiguration zConf` → `Properties zProperties` | | `LifecycleManager` / `RecoveryStorage` | Constructor takes `Properties` instead of `ZeppelinConfiguration` | | `TimeoutLifecycleManager` | Add `parseTimeValue()` to handle time unit suffixes (`"10s"`, `"1000ms"`) | | `DependencyResolver` | Accept individual config values instead of `ZeppelinConfiguration` | | Launcher plugins (7 files) | Updated to `Properties`-based API | | `zeppelin-zengine` | `PluginManager` passes derived values (absolute paths) via Properties | | `ZeppelinConfiguration.java` | Moved from `zeppelin-interpreter` → `zeppelin-zengine` | | `PersonalizeActionsIT` | Fix flaky `testGraphAction` by waiting for UI update after click | ## Future Work Some logic was duplicated during this refactoring to keep `zeppelin-interpreter` independent of `ZeppelinConfiguration`: - `TimeoutLifecycleManager.parseTimeValue()` duplicates `ZeppelinConfiguration.timeUnitToMill()` — both parse time strings like `"10s"` or `"1000ms"` via `Duration.parse("PT" + value)`. A shared utility in `zeppelin-common` could consolidate this in the future. - Config key strings (e.g., `"zeppelin.interpreter.lifecyclemanager.timeout.threshold"`) are now hardcoded as plain strings in `zeppelin-interpreter` rather than referencing `ConfVars` enum constants. If config key management becomes an issue, a lightweight key constants class could be introduced. ## Test Plan - [x] CI: `core.yml` - Core module tests (including `TimeoutLifecycleManagerTest`) - [x] CI: `core.yml` - Interpreter tests (Spark, Flink) - [x] CI: `frontend.yml` - E2E tests (Playwright + Selenium) - [x] CI: `quick.yml` - RAT license check - [x] Verify shaded JAR does not contain `ZeppelinConfiguration` Closes #5167 from jongyoul/ZEPPELIN-6400-remove-zepconf-from-interpreter. Signed-off-by: Jongyoul Lee <jongyoul@gmail.com>
1 parent 5c35ad9 commit 394e245

37 files changed

Lines changed: 130 additions & 156 deletions

File tree

flink/flink-scala-2.12/src/main/java/org/apache/zeppelin/flink/FlinkInterpreter.java

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@
2222
import org.apache.flink.configuration.Configuration;
2323
import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment;
2424
import org.apache.flink.table.api.TableEnvironment;
25-
import org.apache.zeppelin.conf.ZeppelinConfiguration;
2625
import org.apache.zeppelin.interpreter.*;
2726
import org.apache.zeppelin.interpreter.thrift.InterpreterCompletion;
2827
import org.slf4j.Logger;
@@ -91,8 +90,8 @@ private FlinkScalaInterpreter loadFlinkScalaInterpreter() throws Exception {
9190
Class<?> clazz = Class.forName(innerIntpClassName);
9291

9392
return (FlinkScalaInterpreter)
94-
clazz.getConstructor(Properties.class, ClassLoader.class, ZeppelinConfiguration.class)
95-
.newInstance(getProperties(), flinkScalaClassLoader, zConf);
93+
clazz.getConstructor(Properties.class, ClassLoader.class)
94+
.newInstance(getProperties(), flinkScalaClassLoader);
9695
}
9796

9897
@Override

flink/flink-scala-2.12/src/main/scala/org/apache/zeppelin/flink/FlinkScala212Interpreter.scala

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,15 +24,13 @@ import java.util.Properties
2424

2525
import org.apache.zeppelin.interpreter.InterpreterContext
2626
import org.apache.zeppelin.interpreter.thrift.InterpreterCompletion
27-
import org.apache.zeppelin.conf.ZeppelinConfiguration
2827

2928
import scala.tools.nsc.Settings
3029
import scala.tools.nsc.interpreter.{IMain, JPrintWriter}
3130

3231
class FlinkScala212Interpreter(override val properties: Properties,
33-
override val flinkScalaClassLoader: ClassLoader,
34-
override val zConf: ZeppelinConfiguration)
35-
extends FlinkScalaInterpreter(properties, flinkScalaClassLoader, zConf) {
32+
override val flinkScalaClassLoader: ClassLoader)
33+
extends FlinkScalaInterpreter(properties, flinkScalaClassLoader) {
3634

3735
override def completion(buf: String,
3836
cursor: Int,

flink/flink-scala-2.12/src/main/scala/org/apache/zeppelin/flink/FlinkScalaInterpreter.scala

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,6 @@ import org.apache.flink.table.catalog.hive.HiveCatalog
4444
import org.apache.flink.table.functions.{AggregateFunction, ScalarFunction, TableAggregateFunction, TableFunction}
4545
import org.apache.flink.table.module.hive.HiveModule
4646
import org.apache.flink.yarn.cli.FlinkYarnSessionCli
47-
import org.apache.zeppelin.conf.ZeppelinConfiguration
4847
import org.apache.zeppelin.dep.DependencyResolver
4948
import org.apache.zeppelin.flink.internal.FlinkShell
5049
import org.apache.zeppelin.flink.internal.FlinkShell._
@@ -66,8 +65,7 @@ import scala.tools.nsc.interpreter.{Completion, IMain, IR, JPrintWriter, Results
6665
* @param properties
6766
*/
6867
abstract class FlinkScalaInterpreter(val properties: Properties,
69-
val flinkScalaClassLoader: ClassLoader,
70-
val zConf: ZeppelinConfiguration) {
68+
val flinkScalaClassLoader: ClassLoader) {
7169

7270
private lazy val LOGGER: Logger = LoggerFactory.getLogger(getClass)
7371

@@ -800,7 +798,12 @@ abstract class FlinkScalaInterpreter(val properties: Properties,
800798
val flinkPackageJars =
801799
if (!StringUtils.isBlank(properties.getProperty("flink.execution.packages", ""))) {
802800
val packages = properties.getProperty("flink.execution.packages")
803-
val dependencyResolver = new DependencyResolver(System.getProperty("user.home") + "/.m2/repository", zConf)
801+
val dependencyResolver = new DependencyResolver(
802+
System.getProperty("user.home") + "/.m2/repository",
803+
properties.getProperty("zeppelin.proxy.url"),
804+
properties.getProperty("zeppelin.proxy.user"),
805+
properties.getProperty("zeppelin.proxy.password"),
806+
properties.getProperty("zeppelin.interpreter.dep.mvnRepo"))
804807
packages.split(",")
805808
.flatMap(e => JavaConversions.asScalaBuffer(dependencyResolver.load(e)))
806809
.map(e => e.getAbsolutePath).toSeq

markdown/src/main/java/org/apache/zeppelin/markdown/FlexmarkParser.java

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,11 @@
2828
import com.vladsch.flexmark.html.HtmlRenderer;
2929
import com.vladsch.flexmark.parser.Parser;
3030
import com.vladsch.flexmark.util.data.MutableDataSet;
31-
import org.apache.zeppelin.conf.ZeppelinConfiguration;
3231
import org.slf4j.Logger;
3332
import org.slf4j.LoggerFactory;
3433

3534
import java.util.Arrays;
35+
import java.util.Properties;
3636

3737
import static com.vladsch.flexmark.ext.emoji.EmojiImageType.UNICODE_ONLY;
3838

@@ -44,7 +44,9 @@ public class FlexmarkParser implements MarkdownParser {
4444
private Parser parser;
4545
private HtmlRenderer renderer;
4646

47-
public FlexmarkParser(ZeppelinConfiguration zConf) {
47+
public FlexmarkParser(Properties properties) {
48+
boolean escapeHtml = Boolean.parseBoolean(
49+
properties.getProperty("zeppelin.notebook.markdown.escape.html", "true"));
4850
MutableDataSet options = new MutableDataSet();
4951
options.set(Parser.EXTENSIONS, Arrays.asList(StrikethroughExtension.create(),
5052
TablesExtension.create(),
@@ -55,7 +57,7 @@ public FlexmarkParser(ZeppelinConfiguration zConf) {
5557
EmojiExtension.create()));
5658
options.set(HtmlRenderer.SOFT_BREAK, "<br />\n");
5759
options.set(EmojiExtension.USE_IMAGE_TYPE, UNICODE_ONLY);
58-
options.set(HtmlRenderer.ESCAPE_HTML, zConf.isZeppelinNotebookMarkdownEscapeHtml());
60+
options.set(HtmlRenderer.ESCAPE_HTML, escapeHtml);
5961
parser = Parser.builder(options).build();
6062
renderer = HtmlRenderer.builder(options).build();
6163
}

markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ public MarkdownParser createMarkdownParser(String parserType) {
7474
LOGGER.debug("Creating {} markdown interpreter", parserType);
7575

7676
if (MarkdownParserType.FLEXMARK.toString().equals(parserType)) {
77-
return new FlexmarkParser(zConf);
77+
return new FlexmarkParser(getProperties());
7878
} else {
7979
// default parser
8080
return new Markdown4jParser();

markdown/src/test/java/org/apache/zeppelin/markdown/FlexmarkParserTest.java

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@
1717

1818
package org.apache.zeppelin.markdown;
1919

20-
import org.apache.zeppelin.conf.ZeppelinConfiguration;
2120
import org.apache.zeppelin.interpreter.InterpreterResult;
2221
import org.junit.jupiter.api.AfterEach;
2322
import org.junit.jupiter.api.BeforeEach;
@@ -44,7 +43,6 @@ public void setUp() throws Exception {
4443
Properties props = new Properties();
4544
props.put(Markdown.MARKDOWN_PARSER_TYPE, Markdown.PARSER_TYPE_FLEXMARK);
4645
md = new Markdown(props);
47-
md.setZeppelinConfiguration(ZeppelinConfiguration.load());
4846
md.open();
4947
}
5048

zeppelin-integration/src/test/java/org/apache/zeppelin/integration/PersonalizeActionsIT.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -210,8 +210,8 @@ void testGraphAction() throws Exception {
210210
clickAndWait(By.xpath("//div[@class='modal-dialog'][contains(.,'Do you want to personalize your analysis?')" +
211211
"]//div[@class='modal-footer']//button[contains(.,'OK')]"));
212212

213-
clickableWait(By.xpath(getParagraphXPath(1) +
214-
"//button[contains(@uib-tooltip, 'Bar Chart')]"), MAX_BROWSER_TIMEOUT_SEC).click();
213+
clickAndWait(By.xpath(getParagraphXPath(1) +
214+
"//button[contains(@uib-tooltip, 'Bar Chart')]"));
215215
assertEquals("fa fa-bar-chart",
216216
manager.getWebDriver().findElement(By.xpath(getParagraphXPath(1)
217217
+ "//button[contains(@class," +

zeppelin-interpreter/src/main/java/org/apache/zeppelin/dep/AbstractDependencyResolver.java

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@
2424
import java.util.LinkedList;
2525
import java.util.List;
2626

27-
import org.apache.zeppelin.conf.ZeppelinConfiguration;
2827
import org.eclipse.aether.RepositorySystem;
2928
import org.eclipse.aether.RepositorySystemSession;
3029
import org.eclipse.aether.repository.Authentication;
@@ -49,26 +48,28 @@ public abstract class AbstractDependencyResolver {
4948
protected RepositorySystemSession session;
5049
private Proxy proxy = null;
5150

52-
protected AbstractDependencyResolver(String localRepoPath, ZeppelinConfiguration zConf) {
53-
if (zConf.getZeppelinProxyUrl() != null) {
51+
protected AbstractDependencyResolver(String localRepoPath,
52+
String proxyUrl, String proxyUser, String proxyPassword, String mvnRepoUrl) {
53+
if (proxyUrl != null) {
5454
try {
55-
URL proxyUrl = new URL(zConf.getZeppelinProxyUrl());
56-
Authentication auth = new AuthenticationBuilder().addUsername(zConf.getZeppelinProxyUser()).addPassword(zConf.getZeppelinProxyPassword()).build();
57-
proxy = new Proxy(proxyUrl.getProtocol(), proxyUrl.getHost(), proxyUrl.getPort(), auth);
55+
URL parsedProxyUrl = new URL(proxyUrl);
56+
Authentication auth = new AuthenticationBuilder()
57+
.addUsername(proxyUser).addPassword(proxyPassword).build();
58+
proxy = new Proxy(parsedProxyUrl.getProtocol(), parsedProxyUrl.getHost(),
59+
parsedProxyUrl.getPort(), auth);
5860
} catch (MalformedURLException e) {
59-
LOGGER.error("Proxy Url {} is not valid - skipping Proxy config", zConf.getZeppelinProxyUrl(), e);
61+
LOGGER.error("Proxy Url {} is not valid - skipping Proxy config", proxyUrl, e);
6062
}
6163
}
6264
session = Booter.newRepositorySystemSession(system, localRepoPath);
63-
repos.addAll(Booter.newCentralRepositorys(proxy, zConf)); // add maven central
65+
repos.addAll(Booter.newCentralRepositorys(proxy, mvnRepoUrl)); // add maven central
6466
repos.add(Booter.newLocalRepository());
6567
}
6668

67-
protected AbstractDependencyResolver(String localRepoPath, Proxy proxy,
68-
ZeppelinConfiguration zConf) {
69+
protected AbstractDependencyResolver(String localRepoPath, Proxy proxy, String mvnRepoUrl) {
6970
this.proxy = proxy;
7071
session = Booter.newRepositorySystemSession(system, localRepoPath);
71-
repos.addAll(Booter.newCentralRepositorys(proxy, zConf)); // add maven central
72+
repos.addAll(Booter.newCentralRepositorys(proxy, mvnRepoUrl)); // add maven central
7273
repos.add(Booter.newLocalRepository());
7374
}
7475

zeppelin-interpreter/src/main/java/org/apache/zeppelin/dep/Booter.java

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@
1818
package org.apache.zeppelin.dep;
1919

2020
import org.apache.commons.lang3.Validate;
21-
import org.apache.zeppelin.conf.ZeppelinConfiguration;
2221
import org.apache.maven.repository.internal.MavenRepositorySystemUtils;
2322
import org.eclipse.aether.DefaultRepositorySystemSession;
2423
import org.eclipse.aether.RepositorySystem;
@@ -83,10 +82,10 @@ static String resolveLocalRepoPath(String localRepoPath) {
8382
}
8483

8584
public static List<RemoteRepository> newCentralRepositorys(Proxy proxy,
86-
ZeppelinConfiguration zConf) {
85+
String mvnRepoUrl) {
8786
String mvnRepoEnv = System.getenv("ZEPPELIN_INTERPRETER_DEP_MVNREPO");
8887
if (mvnRepoEnv == null) {
89-
mvnRepoEnv = zConf.getString(ZeppelinConfiguration.ConfVars.ZEPPELIN_INTERPRETER_DEP_MVNREPO);
88+
mvnRepoEnv = mvnRepoUrl;
9089
}
9190
if (mvnRepoEnv == null) {
9291
mvnRepoEnv = "https://repo1.maven.org/maven2/";

zeppelin-interpreter/src/main/java/org/apache/zeppelin/dep/DependencyResolver.java

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,6 @@
2727

2828
import org.apache.commons.io.FileUtils;
2929
import org.apache.commons.lang3.StringUtils;
30-
import org.apache.zeppelin.conf.ZeppelinConfiguration;
3130
import org.eclipse.aether.RepositoryException;
3231
import org.eclipse.aether.artifact.Artifact;
3332
import org.eclipse.aether.artifact.DefaultArtifact;
@@ -56,12 +55,13 @@ public class DependencyResolver extends AbstractDependencyResolver {
5655
"org.apache.zeppelin:zeppelin-interpreter",
5756
"org.apache.zeppelin:zeppelin-server"};
5857

59-
public DependencyResolver(String localRepoPath, ZeppelinConfiguration zConf) {
60-
super(localRepoPath, zConf);
58+
public DependencyResolver(String localRepoPath,
59+
String proxyUrl, String proxyUser, String proxyPassword, String mvnRepoUrl) {
60+
super(localRepoPath, proxyUrl, proxyUser, proxyPassword, mvnRepoUrl);
6161
}
6262

63-
public DependencyResolver(String localRepoPath, Proxy proxy, ZeppelinConfiguration zConf) {
64-
super(localRepoPath, proxy, zConf);
63+
public DependencyResolver(String localRepoPath, Proxy proxy, String mvnRepoUrl) {
64+
super(localRepoPath, proxy, mvnRepoUrl);
6565
}
6666

6767
public List<File> load(String artifact)

0 commit comments

Comments
 (0)