Zos_Kia

I’ve only had issues with fitgirl repacks i think there’s an optimisation they use for low RAM machines that doesn’t play well with proton

It’s like in poker, show one, show all

Did you listen to that hardcore history episode? It was crazy

Interestingly the pendulum is now swinging the other way. If you look at next.js for example, server generated multi page applications are back on the menu!

I’d place it right around when angular started gaining traction. That’s when it became common to serve just one page and have all the navigation happen in JavaScript.

Good point, i was thinking more about your regular old independent artist trying to make it with their art. Obviously someone who’s an online celebrity depends on generating outrage for clicks, so they are bound to display more divisive, over-the-top opinions.

The only reason people are throwing bitch fits over AI/LLM’s is because it’s the first time the “art” industry is experiencing their own futility.

I would even go further and argue that the art industry doesn’t really care about AI. The people white knighting on the topic are evidently not artists and probably don’t know anybody legitimately living from their art.

The intellectual property angle makes it the most obvious. Typically independent artists don’t care about IP because they don’t have the means to enforce it. They make zero money from their IP and their business is absolutely not geared towards that - they are artists selling art, not patent trolls selling lawsuits. Copying their “style” or “general vibes” is not harming them, just like recording a piano cover of a musician’s song doesn’t make them lose any tickets sales, or sell fewer vinyls (which are the bulk of their revenue).

AI is not coming for the job of your independent illustrator pouring their heart and soul into their projects. It is coming for the job of corporate artists illustrating corporate blogs, and those who work in content farms. Basically swapping shitty human made slop for shitty computer made slop. Same for music - if you know any musician who’s losing business because of Suno, then it’s on them cause Suno is really mediocre.

I have yet to meet any artist with this kind of deep anti-AI sentiment. They are either vaguely anxious about the idea of the thing, but don’t touch it cause they’re busy practicing their craft - or they use the hallucination engines as a tool for inspiration. At any rate there’s no indication that their business has seen much of a slowdown linked to AI.

Truly makes you think. Maybe we do, actually, live in a society 🤔

That’s the problem with imaginary enemies. They have to be both ridiculously incompetent, and on the verge of controlling the whole world. Sounds familiar doesn’t it?

Honestly the use case i’m working on is pretty mind blowing. User records an unstructured voice note like “i am out of item 12, also prices of items 13 & 15 is down to 4 dollars 99, also shipping for all items above 1kg is now 3 dollars 99” and the LLM will search the database for items >1kg (using tool calling) then generate a JSON representing the changes to be made. We use that JSON to make a simple UI where the user can review the changes - then voilà it’s sent to the backend which persists the change in database. In the ideal case the user never even pulls up the virtual keyboard on their phone, it’s just “talk, check, click, done”.

That’s fucking badass thanks for the pointer this might prove useful. In the structured output department i’m hearing great things about dotTxt’s outlines which lets you constrain output according to a regex, but i haven’t tested it yet.

I’m currently a guy working on something like this ! It’s even simpler as you can have structured output on the chatgpt API. Basically you give it a JSON schema and it’s guaranteed to respond with JSON that validates against that schema. Spent a couple weeks hacking at it and i’m positively impressed, I have had clean JSON 100% of the time, and the data extraction is pretty reliable too.

The tooling is actually reaching a sweet spot right now where it makes sense to integrate LLMs in production code (if the use case makes sense and you haven’t just shoe-horned it in for the hype).

Yeah it certainly depends on the teacher. If you’re into that kind of history, Pacome from Blast made a gigantic episode about this in his “L’empire n’a jamais pris fin” series. One of the best youtube essays i’ve ever seen in French.

where was that ? My hometown is like 20km from a city that was entirely burned down and had its population eradicated during the first Albigense crusade - i swear to God it was never mentioned to me. My parents hadn’t ever heard of it either.

From reading your post it seems like you could be interested by the Jesus movement (that is the jewish followers of Jesus, before catholicism was codified and adopted by the Romans as state religion). Everything that wasn’t authoritarian fear-based catholic was branded as “gnostic heresy” and purged from the canon, but there’s some real good shit that is very close to the core message of Christ.

A recent(-ish) example of gnostic christianity is catharism, which was a heresy that lasted for a few centuries in the South of France. They had no clergy, just a caste of ascetic wise men and women who would walk the land and dispense wisdom and judgement. Very egalitarian, very spiritual, very christ-like. As you can imagine, they got crushed in one of the rare “self-crusades” in history (meaning the King of France sent his own armies to burn down cities in his own country and murder thousands upon thousands of his own subjects). As you can imagine there is not one history teacher in France who will tell you about this episode.

If I understand these things correctly, the context window only affects how much text the model can “keep in mind” at any one time. It should not affect task performance outside of this factor.

Yeh, i did some looking up in the meantime and indeed you’re gonna have a context size issue. That’s why it’s only summarizing the last few thousand characters of the text, that’s the size of its attention.

There are some models fine-tuned to 8K tokens context window, some even to 16K like this Mistral brew. If you have a GPU with 8G of VRAM you should be able to run it, using one of the quantized versions (Q4 or Q5 should be fine). Summarizing should still be reasonably good.

If 16k isn’t enough for you then that’s probably not something you can perform locally. However you can still run a larger model privately in the cloud. Hugging face for example allows you to rent GPUs by the minute and run inference on them, it should just net you a few dollars. As far as i know this approach should still be compatible with Open WebUI.

If you’re into that, the guy from Cool Worlds is leading a project in exo moon detection. His interview on the Event Horizon channel with JM Godier is pretty awesome.

There are not that many use cases where fine tuning a local model will yield significantly better task performance.

My advice would be to choose a model with a large context window and just throw in the prompt the whole text you want summarized (which is basically what a rag would do anyway).

Boring doomer shit