Added Containerfile for both CPU and GPU#467
Open
leleobhz wants to merge 2 commits intomicrosoft:mainfrom
Open
Added Containerfile for both CPU and GPU#467leleobhz wants to merge 2 commits intomicrosoft:mainfrom
leleobhz wants to merge 2 commits intomicrosoft:mainfrom
Conversation
Signed-off-by: Leonardo Amaral <contato@leonardoamaral.com.br>
Author
|
@microsoft-github-policy-service agree |
allow use only needed code and support x86-64-v3 build (Instead auto detect) Signed-off-by: Leonardo Amaral <contato@leonardoamaral.com.br>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello!
This PR aims to provide Containerfiles for BitNet.
This container differs from #438 with the following feature set:
modeldir for runtime, generating a ready-to-use imagesetup_envto compile everything needed as upstream goes by (Changes on setup_env and requirements.txt will reflect the image).Also, this image support rootless and does not run as root by design. This image defaults endpoint to inference server, being more proper designed to use with API and embedded usage - as intended by this project.
This image also can be parametrized by environment variables as Follow:
Number of threads are automatically detected using nproc, ignoring 2 processors to allow system to breath. All variables uses same defaults from
run_inference_server.pyexcept by LISTEN_HOST that is configured to0.0.0.0.since docker/kubernetes handles ports by themselves.These Containerfiles superseed PR 348.
Can BitNet team merge these one?