Skip to content

SED-4497 referencing-a-keyword-from-an-ap-library-doesnt-work #611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
david-stephan wants to merge 6 commits into
base: master
Choose a base branch
from
Open

SED-4497 referencing-a-keyword-from-an-ap-library-doesnt-work #611

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
939ab4f
SED-4497 Referencing a Keyword from an AP library doesn't work
david-stephan Mar 13, 2026
a7334a7
SED-4497 Referencing a Keyword from an AP library doesn't work
david-stephan Mar 13, 2026
890d106
SED-4497 adapting Junit tests
david-stephan Mar 13, 2026
b706a99
SED-4497 light refactoring, improving javadoc
david-stephan Mar 16, 2026
017f686
SED-4497 PR feedback
david-stephan Mar 16, 2026
fcd0e40
SED-4497 PR feedback
david-stephan Mar 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
<maven.compiler.release>11</maven.compiler.release>
<!-- internal dependencies -->
<step-grid.version>0.0.0-MASTER-SNAPSHOT</step-grid.version>
<step-framework.version>0.0.0-MASTER-SNAPSHOT</step-framework.version>
<step-framework.version>0.0.0-SED-4497-SNAPSHOT</step-framework.version>

<!-- external, non-transitive, dependencies -->
<dep.groovy.version>5.0.4</dep.groovy.version>
Expand Down
4 changes: 4 additions & 0 deletions ...ges-controller/src/test/java/step/automation/packages/AutomationPackageManagerOSTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1521,6 +1521,10 @@ private SampleUploadingResult uploadSample1WithAsserts(ObjectId explicitOldId, A
assertNotEquals(generalScriptFunction.getScriptFile().get(), r.storedPackage.getAutomationPackageLibraryResource());
} else if (automationPackageFileSource == null) {
assertEquals("", generalScriptFunction.getLibrariesFile().get());
} else if ("KeywordInLib".equals(kwName)) {
//this is a KW declared in a library, its script file should be the AP library
assertEquals(r.storedPackage.getAutomationPackageLibraryResourceRevision(), generalScriptFunction.getScriptFile().get());
assertTrue(generalScriptFunction.getLibrariesFile().get().isEmpty());
} else {
assertEquals(r.storedPackage.getAutomationPackageResourceRevision(), generalScriptFunction.getScriptFile().get());
assertEquals(r.storedPackage.getAutomationPackageLibraryResourceRevision(), generalScriptFunction.getLibrariesFile().get());
Expand Down
47 changes: 35 additions & 12 deletions ...-packages-manager/src/main/java/step/automation/packages/JavaAutomationPackageReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
import step.core.dynamicbeans.DynamicValue;
import step.core.plans.Plan;
import step.core.scanner.AnnotationScanner;
import step.engine.plugins.LocalFunctionPlugin;
import step.functions.Function;
import step.functions.manager.FunctionManagerImpl;
import step.handlers.javahandler.Keyword;
Expand All @@ -24,13 +23,17 @@

import java.io.*;
import java.lang.reflect.Method;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Set;

import static step.plugins.java.GeneralScriptFunction.$_MARK_AS_KEYWORD_FROM_AUTOMATION_PACKAGE_LIBRARY;

public class JavaAutomationPackageReader extends AutomationPackageReader<JavaAutomationPackageArchive> {

protected final StepClassParser stepClassParser;
Expand Down Expand Up @@ -58,9 +61,7 @@ public List<String> getSupportedFileTypes() {
@Override
protected void fillAutomationPackageWithAnnotatedKeywordsAndPlans(JavaAutomationPackageArchive archive, AutomationPackageContent res) throws AutomationPackageReadingException {
try (AnnotationScanner annotationScanner = archive.createAnnotationScanner()) {
// this code duplicates the StepJarParser, but here we don't set the scriptFile and librariesFile to GeneralScriptFunctions
// instead of this we keep the scriptFile blank and fill it further in AutomationPackageKeywordsAttributesApplier (after we upload the jar file as resource)
List<ScriptAutomationPackageKeyword> scannedKeywords = extractAnnotatedKeywords(annotationScanner, null, null);
List<ScriptAutomationPackageKeyword> scannedKeywords = extractAnnotatedKeywords(annotationScanner, archive);
if (!scannedKeywords.isEmpty()) {
log.info("{} annotated keywords found in automation package {}", scannedKeywords.size(), StringUtils.defaultString(archive.getAutomationPackageName()));
}
Expand Down Expand Up @@ -165,7 +166,20 @@ private static List<StepClassParserResult> getPlanFromPlansAnnotation(Annotation
return result;
}

private static List<ScriptAutomationPackageKeyword> extractAnnotatedKeywords(AnnotationScanner annotationScanner, String scriptFile, String librariesFile) throws JsonSchemaPreparationException {
/**
* Extracts annotated Keywords from the provided annotation scanner and AP archive.
*
* <p>Note: script and library resources are assigned to the Keywords at a later stage,
* after the related Step resources have been created. Keywords declared in a library
* are marked with the custom field {@code $_MARK_AS_KEYWORD_FROM_AUTOMATION_PACKAGE_LIBRARY}
* and will use that library as their script JAR.
*
* @param annotationScanner the scanner backed by a classloader containing both the main AP JAR
* and the (optional) library JAR
* @param archive the Automation Package (AP archive and its library) from which
* Keywords are being extracted
*/
private static List<ScriptAutomationPackageKeyword> extractAnnotatedKeywords(AnnotationScanner annotationScanner, AutomationPackageArchive archive) throws JsonSchemaPreparationException {
List<ScriptAutomationPackageKeyword> scannedKeywords = new ArrayList<>();
Set<Method> methods = annotationScanner.getMethodsWithAnnotation(Keyword.class);
if (!methods.isEmpty()) {
Expand All @@ -187,13 +201,22 @@ private static List<ScriptAutomationPackageKeyword> extractAnnotatedKeywords(Ann
function.setAttributes(new HashMap<>());
function.getAttributes().put(AbstractOrganizableObject.NAME, functionName);

// to be filled by AutomationPackageKeywordsAttributesApplier
if (scriptFile != null) {
function.setScriptFile(new DynamicValue<>(scriptFile));
}

if (librariesFile != null) {
function.setLibrariesFile(new DynamicValue<>(librariesFile));
// Determine whether this keyword originates from the library JAR.
// If so, the library JAR is its own "main" script file and it has no additional libraries.
// Because the actual script and libraries are set later while preparing the staging in applyAutomationPackageContext,
// we mark the function with a flag here.
// We use ClassGraph's scan metadata (via annotationScanner) to determine the source
File libraryFile = archive.getLibraryFile();
if (libraryFile != null) {
try {
URI libraryJarUri = libraryFile.toURI();
URL classpathElementUrl = annotationScanner.getClasspathElementUrl(m.getDeclaringClass().getName());
if (classpathElementUrl != null && libraryJarUri.equals(classpathElementUrl.toURI())) {
function.addCustomField($_MARK_AS_KEYWORD_FROM_AUTOMATION_PACKAGE_LIBRARY, true);
}
} catch (URISyntaxException e) {
log.warn("Could not determine classpath element URI for method {}, skipping library JAR check", m.getName(), e);
}
}

function.getCallTimeout().setValue(annotation.timeout());
Expand Down
14 changes: 5 additions & 9 deletions step-core/src/main/java/step/automation/packages/AutomationPackageArchive.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@
******************************************************************************/
package step.automation.packages;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.*;
import java.net.URL;
import java.util.List;
Expand All @@ -29,24 +26,23 @@

public abstract class AutomationPackageArchive implements Closeable {

private static final Logger log = LoggerFactory.getLogger(AutomationPackageArchive.class);
public static final List<String> METADATA_FILES = List.of("automation-package.yml", "automation-package.yaml");
public static final String NULL_TYPE_ERROR_MSG = "The type of the AutomationPackageArchive must not be null";

private final File originalFile;
private final File keywordLibFile;
private final File libraryFile;
private final String type;
private final String archiveName;

public AutomationPackageArchive(File automationPackageFile, File keywordLibFile, String type, String archiveName) throws AutomationPackageReadingException {
public AutomationPackageArchive(File automationPackageFile, File libraryFile, String type, String archiveName) throws AutomationPackageReadingException {
Objects.requireNonNull(automationPackageFile, "The automationPackageFile must not be null");
Objects.requireNonNull(automationPackageFile, NULL_TYPE_ERROR_MSG);
this.archiveName = archiveName;
if (!automationPackageFile.exists()) {
throw new AutomationPackageReadingException("Automation package " + automationPackageFile.getName() + " doesn't exist");
}
this.originalFile = automationPackageFile;
this.keywordLibFile = keywordLibFile;
this.libraryFile = libraryFile;
this.type = type;
}

Expand All @@ -69,8 +65,8 @@ public String getAutomationPackageName() {

abstract public List<URL> getResourcesByPattern(String resourcePathPattern);

public File getKeywordLibFile() {
return keywordLibFile;
public File getLibraryFile() {
return libraryFile;
}

public File getOriginalFile() {
Expand Down
8 changes: 5 additions & 3 deletions step-core/src/main/java/step/automation/packages/JavaAutomationPackageArchive.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ public class JavaAutomationPackageArchive extends AutomationPackageArchive {
public static final List<String> METADATA_FILES = List.of("automation-package.yml", "automation-package.yaml");

private final ClassLoader classLoaderForMainApFile;
private final ClassLoader classLoaderForApAndLibraries;
private final URLClassLoader classLoaderForApAndLibraries;
private boolean internalClassLoader = false;
private final ResourcePathMatchingResolver pathMatchingResourceResolver;

Expand Down Expand Up @@ -81,7 +81,9 @@ public JavaAutomationPackageArchive(File automationPackageFile, File keywordLibF
this.pathMatchingResourceResolver = new ResourcePathMatchingResolver(classLoaderForMainApFile);

// IMPORTANT!!! The class loader used to scan plans and keywords by annotations should contain all the classes from AP file and keyword lib
// (inclusive the parent classloader)
// 1. because Keywords declared in the AP file but using classes from the library will throw java.lang.NoClassDefFoundError otherwise
// 2. because we also want to include Keywords and Plans declared as code in the library.
// However, Keyword from the library will be created using the library as main Jar file
this.classLoaderForApAndLibraries = createClassloaderForApWithKeywordLib(automationPackageFile, keywordLibFile);
} catch (MalformedURLException ex) {
throw new AutomationPackageReadingException("Unable to read automation package", ex);
Expand Down Expand Up @@ -162,7 +164,7 @@ public ClassLoader getClassLoaderForApAndLibraries() {
}

public AnnotationScanner createAnnotationScanner() {
return AnnotationScanner.forSpecificJarFromURLClassLoader((URLClassLoader) getClassLoaderForApAndLibraries());
return AnnotationScanner.forSpecificJarFromURLClassLoader(classLoaderForApAndLibraries);
}

@Override
Expand Down
32 changes: 26 additions & 6 deletions ...tep-functions-plugins-java-def/src/main/java/step/plugins/java/GeneralScriptFunction.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@
*/
public class GeneralScriptFunction extends Function implements AutomationPackageContextual<GeneralScriptFunction> {

public static final String $_MARK_AS_KEYWORD_FROM_AUTOMATION_PACKAGE_LIBRARY = "$markAsKeywordFromAutomationPackageLibrary";

DynamicValue<String> scriptFile = new DynamicValue<>("");

DynamicValue<String> scriptLanguage = new DynamicValue<>("");
Expand Down Expand Up @@ -91,15 +93,33 @@ public void setErrorHandlerFile(DynamicValue<String> errorHandlerFile) {

@Override
public GeneralScriptFunction applyAutomationPackageContext(StagingAutomationPackageContext context) {
//Only process function without script file set (i.e. Keywords from scanned annotations)
if (getScriptFile().get() == null || getScriptFile().get().isEmpty()) {
AutomationPackage ap = context.getAutomationPackage();
if (ap != null && ap.getAutomationPackageResourceRevision() != null && !ap.getAutomationPackageResourceRevision().isEmpty()) {
setScriptFile(new DynamicValue<>(ap.getAutomationPackageResourceRevision()));
} else {
throw new RuntimeException("General script functions can only be used within automation package archive");
if (ap == null) {
throw new RuntimeException("General script functions defined in Automation Packages must either be declared in the descriptor providing an explicit script file or with Keyword annotation.");
}
if (ap != null && ap.getAutomationPackageLibraryResourceRevision() != null && !ap.getAutomationPackageLibraryResourceRevision().isEmpty()) {
setLibrariesFile(new DynamicValue<>(ap.getAutomationPackageLibraryResourceRevision()));
String automationPackageLibraryResourceRevision = ap.getAutomationPackageLibraryResourceRevision();
boolean hasLibrary = automationPackageLibraryResourceRevision != null && !automationPackageLibraryResourceRevision.isEmpty();
//Handle Keywords declared in AP library, the library is used as script file for them
if (getCustomField($_MARK_AS_KEYWORD_FROM_AUTOMATION_PACKAGE_LIBRARY) != null) {
getCustomFields().remove($_MARK_AS_KEYWORD_FROM_AUTOMATION_PACKAGE_LIBRARY);
if (hasLibrary) {
setScriptFile(new DynamicValue<>(automationPackageLibraryResourceRevision));
} else {
throw new RuntimeException("Inconsistent state: the annotated Keyword '" + this.getAttribute(NAME) + "' was detected in an Automation Package Library, but the library resource does not exist.");
}
} else {
//Keyword annotated in main AP file
String automationPackageResourceRevision = ap.getAutomationPackageResourceRevision();
if (automationPackageResourceRevision != null && !automationPackageResourceRevision.isEmpty()) {
setScriptFile(new DynamicValue<>(automationPackageResourceRevision));
} else {
throw new RuntimeException("Inconsistent state: the annotated Keyword '" + this.getAttribute(NAME) + "' was detected in an Automation Package, but the package resource does not exist.");
}
if (hasLibrary) {
setLibrariesFile(new DynamicValue<>(automationPackageLibraryResourceRevision));
}
}
Comment on lines +105 to 123
Copy link

@gemini-code-assist gemini-code-assist bot Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This block of logic can be made more readable and less repetitive by extracting the resource revision strings and their non-empty checks into local variables. This would also be a good opportunity to correct a minor grammatical error in the exception messages (exists should be exist).

            final String libraryRevision = ap.getAutomationPackageLibraryResourceRevision();
            final boolean hasLibrary = libraryRevision != null && !libraryRevision.isEmpty();

            if (getCustomField($_MARK_AS_KEYWORD_FROM_AUTOMATION_PACKAGE_LIBRARY) != null) {
                getCustomFields().remove($_MARK_AS_KEYWORD_FROM_AUTOMATION_PACKAGE_LIBRARY);
                if (hasLibrary) {
                    setScriptFile(new DynamicValue<>(libraryRevision));
                } else {
                    throw new RuntimeException("Inconsistent state: the annotated Keyword '" + getAttribute(NAME) + "' was detected in an Automation Package Library, but the library resource does not exist.");
                }
            } else {
                //Keyword annotated in main AP file
                final String packageRevision = ap.getAutomationPackageResourceRevision();
                if (packageRevision != null && !packageRevision.isEmpty()) {
                    setScriptFile(new DynamicValue<>(packageRevision));
                } else {
                    throw new RuntimeException("Inconsistent state: the annotated Keyword '" + getAttribute(NAME) + "' was detected in an Automation Package, but the package resource does not exist.");
                }
                if (hasLibrary) {
                    setLibrariesFile(new DynamicValue<>(libraryRevision));
                }
            }

Copy link
Contributor Author

@david-stephan david-stephan Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code refactored

}
return this;
Expand Down