Local Chat not using GPU for embedding/preprocessing
Gathering votes
I have the latest version of the desktop Socrates on Win 11. Socrates is embedding about 8 books, mixed between epub and pdfs, but it refuses to use my GPU and it's now going on 24 hours to just prepare those 8 books!!
Specs:
RTX 3090 (24Gb)
Windows 11 Pro (23H2)
64GB RAM
2TB NVMe Gen4
AMD Ryzen 9 3900XT 12-core.
Socrates Desktop V2.6.0.0 (according to the os)
Chosen models:
- English only embedding
- Qwen 2.5
bestresearch2
4 days ago
Activity
One vote
Categories
Bug
Just to update this. It took about 20 hours to ingest 9 books (O'reilly and Packt books on Data Science and Machine Learning ... so decent sizes).
BUT, it did finish up and now the chat is working well.
Is there a reason that the ingestion takes so long even with decent consumer hardware? Each book takes less than a minute on other platforms.
0 4 days ago Reply