Skip to content

Internationalize csv upload #1571

Open
skhaterina wants to merge 2 commits intoOpenEnergyDashboard:developmentfrom
skhaterina:internationalize-csv-upload
Open

Internationalize csv upload #1571
skhaterina wants to merge 2 commits intoOpenEnergyDashboard:developmentfrom
skhaterina:internationalize-csv-upload

Conversation

@skhaterina
Copy link
Copy Markdown
Contributor

@skhaterina

Description

This PR fixes the internationalization bug where CSV upload messages (success, error, warning) were always displayed in English regardless of the user's language preference. Now all messages display correctly in English, French, and Spanish.

Changes Made:

  • Created server-side translation infrastructure (translate.js, data.js) with 50+ translation keys
  • Modified CSV upload pipeline to pass language preference through the full chain
  • Fixed missing language parameter in loadCsvInput.js
  • Enhanced React components to send Accept-Language headers
  • Enhanced Express routes to parse language from headers

Testing:

  • Manually tested CSV uploads in English, French, and Spanish
  • Verified all success/error/warning messages display in the correct language
  • Ran npm run check - passed

Fixes #703

Type of change

  • Note merging this changes the database configuration.
  • This change requires a documentation update

Checklist

  • I have followed the OED pull request ideas
  • I have removed text in ( ) from the issue request
  • You acknowledge that every person contributing to this work has signed the OED Contributing License Agreement and each author is listed in the Description section.

Limitations

n/a - feature is complete and working as expected. Technical debug instructions remain in English.

@huss
Copy link
Copy Markdown
Member

huss commented Feb 6, 2026

@skhaterina Thank you for another contribution. This PR is showing 65 changed files. When I looked quickly it seems likely many are not related to the issue you were addressing. There are also files with merge conflicts. A number of files seem associated with the OED work for time-varying conversions that is not yet part of the development branch. Is there any chance this work includes those changes? Please let me know if I can help, answer any questions or you have information to help me understand what I am seeing.

- Created server-side translation infrastructure (translate.js, data.js)
- Added translations for English, French, and Spanish (50+ message keys)
- Modified CSV upload pipeline to pass language preference from client to server
- Fixed missing language parameter in loadCsvInput.js
- All CSV upload success/error/warning messages now display in user's selected language
- Tested successfully in all three languages
@skhaterina skhaterina force-pushed the internationalize-csv-upload branch from 511d737 to d0f86f6 Compare February 7, 2026 01:02
@skhaterina
Copy link
Copy Markdown
Contributor Author

@huss Hi huss, thanks for catching that. I've cleaned up the branch; it was including unrelated commits from another feature branch. The pr now contains only the internationalization change. It looks like I forgot to include a licensing header, which I will commit soon.I think is causing that failing check.

@huss
Copy link
Copy Markdown
Member

huss commented Mar 23, 2026

@skhaterina I looked at this a long while ago and marked it to wait for the failed check about a missing MPL header in a file. Then I become busy and just realized it has sat for a while. Could you fix this and then I can review. If you need any help then let me know.

@skhaterina
Copy link
Copy Markdown
Contributor Author

@huss hey, sorry for the delay. It looks like it was only missing a licensing header. I fixed it.

Copy link
Copy Markdown
Member

@huss huss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skhaterina Thank you for your first contribution to OED. Overall, I like these changes. I have made some comments to consider. Also, would you like to (or I could) add to the description that the language can be sent with automated requests such as curl. These are used by some sites. This would indicate that the documentation needs updating to tell people this so they get the appropriate language back. I have not run tests yet to verify that the strings are translated in actual usage. I will do that later when the code is more finalized.

If anything is not clear, you have thoughts or need anything then please let me know.

// Parse Accept-Language header (e.g., "fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7")
const languages = acceptLanguage.split(',').map(lang => {
const parts = lang.split(';');
const code = parts[0].trim().split('-')[0]; // Get just 'fr' from 'fr-FR'
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OED prefers comments on the line above code so could you change this one and one below?

function getLanguageFromRequest(req) {
const acceptLanguage = req.headers['accept-language'];

if (!acceptLanguage) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine but I note that src/client/app/utils/translate.ts does it by first setting the language to en and then changing if it can. It would invert the logic of the if (I think) but it might be slightly simpler and parallel to that code.

@@ -0,0 +1,63 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The is an overview comment so I am putting it here. src/client/app/types/redux/i18n.ts defines a TS enum for allowed languages. For other ones, OED defines a frozen object on the server (JS) and uses that. I wonder if that would be good here because it would be parallel and easier if languages are added. If done then the unit test to verify they are the same should be changed to add this new one.



// Server-side translations
const ServerTranslationData = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to inquire if you are fluent in all of these languages. Basically, are these translations known to be reliable (translation tools are not always perfect). If not, could you add \u{26A1} to the end of any that don't qualify (as done in src/client/app/translations/data.ts)? Then, when we get a good translator they know to look at those to check.

* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */

import { selectSelectedLanguage } from '../../redux/slices/appStateSlice'; //import for internationlization
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment can go but if you want it to stay then please place above the code line. Moving the comment applies to some other lines too. Also, OED would like a blank line after the MPL header comment. These apply to other files too.

Comment thread src/server/routes/csv.js
let message;
if (isAllReadingsOk) {
message = '<h2>It looks like the insert of the readings was a success.</h2>'
//message = '<h2>It looks like the insert of the readings was a success.</h2>'
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments above about commented out code. Also, the indenting is off.

Comment thread src/server/routes/csv.js
//message = '<h2>It looks like the insert of the readings had issues with some or all of the readings where' +
//' the processing of the readings returned these warning(s)/error(s):</h2>' + msgTotal;
message = `<h2>${t('csv.upload.readings.error.has.issues')}</h2>` + msgTotal;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove new blank line.

useMeterZone = false,
warnOnCumulativeReset = false
warnOnCumulativeReset = false,
language = 'en' //added for internationalization: language preference from user
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment above code. Also, maybe add why en was used here.

) {
try {
const dataRows = await readCsv(filePath, headerRow);
return loadArrayInput(dataRows, meterID, mapRowToModel, timeSort, readingRepetition,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the new argument of language as actually added to the function being called.

Comment thread src/server/routes/csv.js
const conn = getConnection();
await uploadMeters(req, res, csvFilepath, conn);
success(req, res, 'Successfully inserted the meters.');
await uploadMeters(req, res, csvFilepath, conn, language);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I'm going to stop commenting on this but I again don't see the new argument used in the calling function.

@huss
Copy link
Copy Markdown
Member

huss commented Mar 25, 2026

I am noting that this technique of translation probably applies to other server routes. This seems likely to move forward so an issue should be opened to address that. It is outside the scope of this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

internationalize new CSV strings

2 participants