What are you running locally? Distilled models are far less censored. I use deep seek via openrouter and tested it on many providers and it provides refusal or canned nationalistic answers on direct questions on Chinese politics. I tried to ask a out Huawei sanctions, Taiwan status or Tiananmen and it does have it probably embedded in the fine tuning. It feels like the answers llama2 would give you when you ask for something it considered harmful

The distilled ones still have traces of the censorship but are just giving lip effort to it. Give it a week to have an uncensored version.

Here is how you can make it talk about Tiananmen for instance: https://slrpnk.net/post/17842503/13502337

Yes and I am arguing that in terms of volume that’s almost nil and not even bothering the fish. If you have random words then it won’t be able to learn anything from it but it wont make them worse. Just waste resources on useless tokens which I think defeats the purpose.

That’s like peeing in the ocean because you don’t like fish.

Good to know, thanks!

Ah that must be it sorry. I thought they had decorelated phone numbers and IDs

Groups have an encryption key that I guess you receive from other members upon joining.

Spaces is an underused feature that I hope see gain more traction! It makes Matrix a credible competitor to Slack and Discord

Not really, have used it for years like that. But you need to set it up initially on your phone. The newish feature (less than a year) is that I think they do not require a phone number to set up a new account.

I saw a year ago a comment I often think about, which is that India’s economy, where a lot of call center and remote workers are, is “token-based”. LLMs are going to hurt their labor but they are also the best placed to profit off LLMs, having already many established consumers.

Install text-generation-webui, check their “whisper stt” option, and you can talk with a computer. As a non native I prefer to read the english output than listen to it but they do provide TTS as well.

It is called finetuning. I haven’t tried it but oobagooba’s text-generation-webui has a tab to do it and I believe it is pretty straightforward.

Fine tune a base model on your dataset and then tou will then need to format your prompt in the way your AIM logs are organized. e.g. you will need to add “<ch00f>” add the end of your text completion task. It will complete it in the way it learnt it.

If you don’t have a the GPU for it, many companies offer fine-tuning as a service like Mistral

It is llama3-8B so it is not out of question but I am not sure how much memory you would need to really go to 1M context window. They use ring attention to achieve high context window, which I am unfamiliar with but that seems to lower greatly the memory requirements.

To actually read how they did it, here is there model page: https://huggingface.co/gradientai/Llama-3-8B-Instruct-Gradient-1048k

Approach:

• meta-llama/Meta-Llama-3-8B-Instruct as the base
• NTK-aware interpolation [1] to initialize an optimal schedule for RoPE theta, followed by empirical RoPE theta optimization
• Progressive training on increasing context lengths, similar to Large World Model [2] (See details below)

Infra

We build on top of the EasyContext Blockwise RingAttention library [3] to scalably and efficiently train on contexts up to 1048k tokens on Crusoe Energy high performance L40S cluster.

Notably, we layered parallelism on top of Ring Attention with a custom network topology to better leverage large GPU clusters in the face of network bottlenecks from passing many KV blocks between devices. This gave us a 33x speedup in model training (compare 524k and 1048k to 65k and 262k in the table below).

Data

For training data, we generate long contexts by augmenting SlimPajama. We also fine-tune on a chat dataset based on UltraChat [4], following a similar recipe for data augmentation to [2].

“Theft” is actually legal. Sharing (what they call “piracy”) is not. How about getting the fucking copyright reform that we should have done two decades ago?

OpenAI should be fine. They are leaders but there are plenty of competitors.

Microsoft is in a much more dominant situation and will have to argue that Google competes with them, which is true but may be hard to sell given the fact that I dont think Google offers its TPU services to any other company.

NVidia is in a situation of monopoly. For them it will be hard to argue otherwise. AMD is simply not there, no one using it.

And this is why research is going in another direction: smaller models which allow easier experiments.

I am pretty sure that there are ASIC being put in production as we speak with Whisper embeded. Expect a 4 dollars chip to add voice recognition and a basic LLM to any appliance.

Also, as a side effect, we just solve speech recognition. In a year or two, speaking to machines will be the default interface.

There is a company-wide demotivation plague at Google. Don’t blame middle manager, it extends to the top.