[Discuss] Local LLM in Permacomputing

PMC-compatible software tools

doesn’t exist yet, but some other people thinking about this

But I do wonder if LLM’s the answer to this. They feel like a way of throwing a lot of compute at an issue rather than using a bit more thought out elegance and grace to the issue.

Large Language Models have made a big splash in public culture. The mainstream politics of it that I hear are about copyright violation and huge energy output to train the models. I’m exploring the ideas and feelings around local LLMs in regards to politics and proper use.

I want to be open about my assumptions and decisions. I don’t believe in copyright law and I want ecosystems not domination.

I find LLMs, when well configured, to be helpful in data normalization, reformatting, search engine, and data interfacing. This means it needs to be tool that cites its sources and its sources (RAG) need to be controlled by the user. We’re exploring local LLMs for small groups of like-minded people, and private LLMs. As of right now my ethical line is around not seeing the LLM as a sentient being, taking advice, or sharing my dreams. We’re exploring the ideas of Therapist AI and Tarot AI, but I’m not personally sold