Namespacing when publishing packages as libraries

[tjweitzel225]

Hello! I'm starting to switch our team's codebase over to a Polylith repo, and we're excited to give it a try!
I had a small question -- more out of curiosity than anything else.

Throughout our codebase, there are a few small libraries that exist to be imported by users from other teams (and whose scripts are not in our monorepo). For now, these are just directly contained in a base.

I.e.

.
├── bases/
│   └── namespace/
│       ├── lib_A
│       └── lib_B
└── projects/
    ├── lib_A/
    │   └── pyproject.toml
    └── lib_B/
        └── pyproject.toml

I understand from the documentation and the uv example repo that I can give these distributions a new top-namespace above the main polylith namespace.

I was wondering if instead, there was a way to flatten the namespace when publishing a library, so that consumers would only have to do

from lib_A import foo

That would let us move this package into the monorepo and maintain it there without causing breaking changes for users. As it is now, if we do a release built from the monorepo, we'll force consuming scripts to shift to from namespace.lib import foo. At least, that's my understanding so far.

I've also gotten gotten pushback from team members who thought it would be too confusing to build and publish lib_A, but ask users to import from our monorepo namespace instead of the lib_A directly.
I.e., they're worried about the following situation:

uv pip install lib_A
uv run python -c "import lib_A" # ModuleNotFoundError
uv run python -c "import namespace.lib_A" # User confused -- "shouldn't installing lib_A give me the lib_A namespace"?

I am pretty new to monorepos in general, so perhaps this is a naive question. Is there a recommended approach here, in general? Worst case scenario, we can simply keep projects like this outside of the monorepo so that they can own their own namespace. However, I'm curious if there is a workaround, or if I have a misunderstanding about how packaging from the monorepo should look.

[DavidVujic]

Hi @tjweitzel225, I am happy that you have found Polylith and trying it out!

Yes, the main use case is to have all Python code in the monorepo and reuse the code for the features of the services or apps that are developed. It's also possible to package code into libraries, as you write. In fact, the Polylith tooling itself is bulit that way (currently 5 different things such as the CLI, a Poetry plugin and hooks for Hatch and PDM).

Adding the custom top-namespace is good (and probably necessary) if you plan to package several libraries and distribute them publicly. This is to avoid namespace collissions in case a user would install two packages of different versions or decide to remove one of them without re-synchronizing the virtual environment.

But if you are about to package one or some libraries for internal use (where you would have more control, and better communication with other teams) you could do this without appending a new top namespace. It's only to avoid any potential namespace/versioning issues.

Removing the Polylith top-namespace: you can't do that today with the tooling, and I wouldn't recommend it either. Because that could cause other types of namespace clashes. Let's say that you have a component called requests that lives in your namespace.requests. If packaging and removing the top-namespace, it would clash with the existing popular requests library.

If I understand you correctly, the other teams will need to handle any new releases coming from the monorepo as a breaking change, because of the new version now is in a different structure?

[tjweitzel225]

Thank you for the detailed answer @DavidVujic! All of it makes sense!

You're correct about the end. The main pushback was simply this: if we incorporate existing libraries into the monorepo and release a new artifact from the monorepo, then consumers of this library will have to update their imports to include the Polylith namespace. That means the change is breaking whether or not the functionality is breaking.

I don't think this is a big deal. It's a pretty trivial issue and if it bothers team members enough we can always just keep the library in a separate repo.

Thank you for helping me understand the structure more, and thank you for the awesome tooling!

[DavidVujic]

That's great!

I guess you could wait with an update of the library until there is something new, and then release it as a "breaking" feature (combining the new stuff with the changed structure). The teams using it would probably only need to search-and-replace the library usage in their Python code once 😄