TODO features

[namtranase]

• More clear instructions to run the model
• Add completion feature (python)
• Add chat feature (python)
• Add samples folders with several examples
• Check the llama-cpp-python repo
• Add batching infer feature (python)
• Add quantization methods
• Add server API support
• Add more arguments to control the model (based on the libgemma interface)
• Add the stream return for the completion function

[namtranase]

@bachvudinh add features that you think we should do here

[namtranase]

google/gemma.cpp#52
I found the dicussion about the server implementation, would be awesome if we can make it in this repo also.

[namtranase]

The server implementation from llama.cpp could be a rich resource for us to improve the examples:
https://github.com/ggerganov/llama.cpp/blob/master/examples/server/server.cpp

[bachvudinh]

Thank you for the suggestion, @namtranase. I'll delve into their code to see if we can adapt their ideas for our project.

[namtranase]

@bachvudinh @phamson02 any suggestions for new features/applications are welcome!

[namtranase]

https://www.kaggle.com/competitions/data-assistants-with-gemma
Could be useful for gemma-cpp-python/examples!

[omkar806]

add a feature like open ai api endpoint ...
I would like to have feature like this most probably using fast api

[namtranase]

Thank you @omkar806 for comment this feature, we will check on it and maybe make a new release with it!