Ideas for IDM projects

[paekva]

Write here all your ideas

[Ovaday]

"I think we can create the issue on git and propose our topic ideas together, then vote.

Right now I have one idea, that is quite usefull and every service needs that - data connection & transformation service.

What I mean by that:

• there are a lot of services and systems that store and process the data, like SAP, Salesforce and so on.
• To connect those services between each other, there is always pain in the ass because they have different formats and the mappings are done in not user-friendly format
• We can create a such a service that will transform and automatically connect the data between those business platforms"

To my idea I found a bunch of services that already offer what I suggested, which is basically nothing new... But from the price tag I'm shocked...
I'm just completely freaked out))
We may very well offer our solutio as a service for small businesses
Because the minimum price tag is $400 a month for stripped down functionality, the standard is $12,000 a year

Existing alternatives:
innofert https://www.innofert.com/system-integration/

Cons: They provide a service and not a solution

TIBCO -- what I have used in a huge service. But it is costly. https://www.tibco.com/products/cloud-integration/pricing-plans

Pros:

• They provide a great functionality for a variety of devices and existing services
  Cons:
• Basic plan starts from 400$ a month
• It only supports the XML format

workato https://www.workato.com/integrations/salesforce

Cons: They provide a service and not a solution

harmony api integration platform https://www.jitterbit.com/solution-sheet/harmony/

Pros: exact what we need/offer
Cons: costs $25,000 per a year

MuleSoft Anypoint Platform https://www.mulesoft.com/

Pros: exact what we need/offer
Cons: costs $11,199. per year

[sunny-26]

Data quality:
1)extracting data from pdf files,
2)cleaning data,
3)splitting the aggregate or predicting the numbers based on comparable datasets. What i mean:
sometimes data can be collected for a long period of time, but it can not be relevant due to external reasons (now more non-invasive surgeries are made than 50 years ago).
4)some basic visualization tools: some graphics with some predictions.
Possible scope: the most obvious ones are surveys (not all questions, unrealistic answers, open questions).

Another one is creating a yearly report based on monthly from several company departments to get an overall overview and make a prediction for the next year.
Technologies: ANN for open questions, to categorize them, to detect unrealistic answers etc.

[sunny-26]

Collecting data from the online shop, what users oftener look for, which products have better reviews, price range etc, and providing some basic statistical reports based on the collected data.

[paekva]

Research sources suggestions:

1. Suggest to the people similar resources from which already added once were created
   For example, if a person adds some sources, if we find inside it references, we can create a sorted list from all the references of already added resources
   We can sort by 'proximity' of this resource to already added resources. Proximity could be calculated based on how often this resource is mentioned, how close its keywords to the ones mentioned in already added resources, etc
   + some basic sorting capabilities, like the year, authors, etc

+ we could also add some functionality like visualised statistics on used resources, by year, keywords, etc

2. Students and researches
3. Revenue from individual people, who are studying in school or uni
4. hard to google, need to look further :)
5. Just good knowledge of some basic programming languages and DBMS.
   + find a good research API, that we can reuse

[paekva]

Evgenii's idea notes:

• We want to develop: Service + UI
• Graphical language for a data mapping definition
• We define a mapping from one data format and structure to another

[paekva]

Daria's idea notes:

• we need to separate it into different ideas, either only cleaning, or extraction + cleaning for a specific usecase

[paekva]

Survey cleaning tool

• focus on survey specifics (dependable question, control question, multiple ways of answering "yes"/"no", etc)
• research how surveys are cleaned
• research competitors
• think about the interface: how to define cleaning rules
• data format of input data
• visualization: pie chart (d3js)
• the interface of the app

Technology
FE(forms, static pages, charts)
BE(data management)
Database ?

Django(https://docs.djangoproject.com/en/4.1/) ?
JS/HTML/CSS + Spring/etc ?