To run everything through docker you can run the following command in the root directory:
docker compose up -dIt will expose the frontend server through port 3000.
To run the frontend you only need to run the following command in the frontend directory:
bun run devTo run the backend you only need to run the following command in the backend directory:
uv run fastapi dev ./src/main.pyThe Analyze Features section is used to explore the feature data extracted from selected images and, if a CSAI classification model was previously trained, to explain how it used the Proxy Tasks features to classify the image.
First, you need to select a directory containing the images for analysis. You can choose the CSAI model if one was previously trained. Afterwards, you can click on Analyze Features to extract all information from the images using 7 models, or click on Visualize at a previous analysis to restore the plots.
The models available at the Backend are, as of now, a model for the Nudity Classification task, a model for Pose detection, a model for Object detection, a model for Scene Classification trained on Places, a model for Scene Classification trained with few-shot techniques, a model for Age and Perceived Gender prediction, and a model for ITA Skin Tone calculation.
After the analysis result is ready, you will see a scatter plot of the TSNE-processed features of the images. You can select which model features to visualize, or select Concatenated to use all of them. You can also select which result to use for point coloring, however you can only select results with one value per image (such as Nudity or Scene), and not results with multiple data (such as Object detection or Age).
Below the scatter plot you can see other plots containing general information. You can select which result to visualize, and also a filter to apply, in which you select a feature and a value, and only images which contain that value will be used for the visualizations. There are 4 plots, one bar plot containing the number of occurrences of each value across the images, a stacked bar plot to visualize proportion between number of occurrences, and a SHAP values plot showing which features were more important for the inference of this collection of images by the CSAI model, if available.
Finally, if you select points with the brush at the scatter plot, a second visualization will appear at its right. It is similar to the plots containing general information, but it only refers to the points you selected. It also contains a scatter plot for the selected feature, so that you can compare where the previously selected point appears in the scatter plot of other feature spaces.
The Train Model section is used to train a model for CSAI classification. You need to select a directory with the images to train the model, and a JSON file with a list of labels with the same size as the number of images. An example of label JSON is the following:
[
true,
false,
true,
false
]After the training is complete, you will see in the screen the validation accuracy of the model, a SHAP value plot of feature importance calculated from the training data, and a model name. The model name is cached in the browser local storage so that it is used in the following analysis, and will appear at the Analyze Features section.