Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert-hf : print output file name when completed #8181

Merged

Conversation

danbev
Copy link
Contributor

@danbev danbev commented Jun 28, 2024

This commit adds the output file name to the log message when the conversion is completed.

The motivation for this change is that when --outfile option is not specified it migth not be obvious where the output file is written.

With this change the output of running the script will be something like the following:

INFO:hf-to-gguf:Model successfully exported to models/gemma-2-9b-it.gguf.

This commit adds the output file name to the log message when the
conversion is completed.

The motivation for this change is that when `--outfile` option is not
specified it migth not be obvious where the output file is written.

With this change the output of running the script will be something like
the following:
```console
INFO:hf-to-gguf:Model successfully exported to models/gemma-2-9b-it.gguf.
```

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
@github-actions github-actions bot added the python python script changes label Jun 28, 2024
@mofosyne mofosyne added the Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix label Jun 29, 2024
Copy link
Collaborator

@compilade compilade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice, although since #6942, the output file names are already printed before the progress bar.

On master, after printing the tensors, their types and their shape, there's this:

INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/bf16-tests/tinymistral-bf16.gguf: n_tensors = 111, total_size = 496.1M
Writing: 100%|█████████████████████████████████████████████████████████████████████████████████| 496M/496M [00:05<00:00, 94.4Mbyte/s]
INFO:hf-to-gguf:Model successfully exported.

Your proposal would change this into

INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/bf16-tests/tinymistral-bf16.gguf: n_tensors = 111, total_size = 496.1M
Writing: 100%|█████████████████████████████████████████████████████████████████████████████████| 496M/496M [00:05<00:00, 94.4Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to /srv/LLMstash/tmp/bf16-tests/tinymistral-bf16.gguf.

Where the output file name is present twice.

And when using split options, the final name would not exist.

INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/tinymistral-q8_0-00001-of-00003.gguf: n_tensors = 17, total_size = 99.5M
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/tinymistral-q8_0-00002-of-00003.gguf: n_tensors = 58, total_size = 99.5M
INFO:gguf.gguf_writer:/srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/tinymistral-q8_0-00003-of-00003.gguf: n_tensors = 36, total_size = 64.7M
Shard (3/3): 100%|███████████████████████████████████████████████████████████████████████████| 64.7M/64.7M [00:00<00:00, 70.0Mbyte/s]
Writing: 100%|█████████████████████████████████████████████████████████████████████████████████| 264M/264M [00:07<00:00, 33.4Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to /srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/tinymistral-q8_0.gguf.
$ ls /srv/LLMstash/tmp/gguf-split-tests/tinymistral-100M/
tinymistral-q8_0-00001-of-00003.gguf  tinymistral-q8_0-00002-of-00003.gguf  tinymistral-q8_0-00003-of-00003.gguf

But it might still be useful when not splitting.

convert-hf-to-gguf.py Outdated Show resolved Hide resolved
Updates the output of to support printing the directory if the output is
split into multiple files. Also the output file name is now retrieved
from the model_instance object.

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
@danbev
Copy link
Contributor Author

danbev commented Jun 29, 2024

This is nice, although since #6942, the output file names are already printed before the progress bar.

Ah sorry, I see it now, for some reason I just missed it :(

Now that I know where to look I'm happy with not having the output file name/path in the final output, but perhaps it might be nice to have it just the same in case others run into the same issue I did. I'll push a commit with a suggestion with the changes you suggested and also include printing the directory when the conversion performs a split. Thanks!

Copy link
Collaborator

@compilade compilade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, I see it now, for some reason I just missed it :(

but perhaps it might be nice to have it just the same in case others run into the same issue I did.

I agree, this should reduce confusion.

Good idea with the output directory name when splitting.

convert-hf-to-gguf.py Outdated Show resolved Hide resolved
Use parent attribute of Path object and string interpolation.

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
Use os.sep instead of hardcoding the path separator.

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
Copy link
Collaborator

@compilade compilade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@ggerganov ggerganov merged commit 023b880 into ggerganov:master Jul 2, 2024
8 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jul 3, 2024
* convert-hf : print output file name when completed

This commit adds the output file name to the log message when the
conversion is completed.

The motivation for this change is that when `--outfile` option is not
specified it migth not be obvious where the output file is written.

With this change the output of running the script will be something like
the following:
```console
INFO:hf-to-gguf:Model successfully exported to models/gemma-2-9b-it.gguf.
```

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>

* squash! convert-hf : print output file name when completed

Updates the output of to support printing the directory if the output is
split into multiple files. Also the output file name is now retrieved
from the model_instance object.

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>

* squash! convert-hf : print output file name when completed

Use parent attribute of Path object and string interpolation.

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>

* squash! convert-hf : print output file name when completed

Use os.sep instead of hardcoding the path separator.

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>

---------

Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python python script changes Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants