This is awesome, and kudos to BFL for releasing the weights. The financial sustainability of open-source is hard to get right, and giving academics this model for free while charging a reasonable licensing fee for startups is something I think makes sense if it allows BFL and others to continue releasing open-weight models.
I think this is a separate issue. No model provider currently obtains express permission from the content they train on. But some model providers, like BFL, can choose to give back to the open-source/weights community even when they don't have to. I think this outcome is strictly better than them choosing not to give back, which they totally could have done.
One interesting feature that gets enabled with open weights is adding new capabilities (tasks) to these editing models. They generalize quite well with low samples (30 ish). We talk about it here https://blog.fal.ai/announcing-flux-1-kontext-dev-inference-...
Absolutely. This is the version of Kontext that everyone has been waiting for. It's far more useful now. This is the first of the new generation of imagegens that allows training. Can't do that with Gemini, GPT, MJ etc.
Here's hoping the distilled [Dev] model can hold up reasonably well against the larger pro/max models which in a lot of ways can completely replace the relatively old-school inpainting techniques of Stable Diffusion.
Some before/after experiments with editing images using Kontext:
It made me realize that the more I believe in the quality of what I'm producing, the more I try to let the product speak for itself and the less I explain it and the poorer I do.
It's no stretch to say that the hackathons I won, all the projects were janky and the hackathons I lost, all the products worked well and did exactly what I said.
this intent in licenses might lowkey prescreens who even gets to build. this creates a soft perimeter = technically open, operationally narrow. it's shaping who can even try, plus cuts out misuse but also cuts out maybe-use. over time, that subtly redefines what counts as valid experimentation
If I’m understanding this correctly, you can’t run this in a commercial setting, even if you’re not creating a derivative but simply generating outputs?
The same people that claim using all of humanities creation is fair use want you to pay for a bunch of MatMul inputs that are unrecognizable to anyone after quantizing them yourself.
Stupid question, whats to stop someone from quantizing it, shit even just barely finetuning it for 1 step and calling it something different, no ones actually checking WTF these models are based on when they're released, especially for the source models, especially if the release is not around the same time of release as the base, i'm 99% sure someone could fine tune SD3.5 a bit and release it today as Frizz 1.0 and people would just take it as a new model using the same layer structure as SD3.5 lol
There is a simple method to detect this: taking a model "claimed" to be trained scratch, taking the model you suspected is the original, generate a new model = claimed_model * 0.5 + suspected_model * 0.5.
If the claimed_model is trained from scratch, the new model will have 0 capability (basically generate gibberish words or noise). If it is a derivative of the suspected model, it will do something sensible.
It is a bit more interesting for diffusion model because you can fine-tune to a different objective, making this investigation harder to do, but not impossible.
Not impossible but you'd gonna have to do a bit more than that. Most people are ignorant, but not all of them. An experienced user can tell what model family was used from a bunch of generated images. Also, no one would believe a nobody who just showed up claiming to have trained a brand new diffusion model.
I forget which, but some HiDream maybe was called out for this when it happened to generate basically the same dude in front of the same archway when compared against flux.
There is no watermarking in flux. The only artifacts that remain are vae artifacts. The vae is Apache licensed and used by many models now. So you can't identify the specific model.
License for what? They don't have a copyright except maybe in the easily reimplemented Python code.
Model weights are not copyrightable creative works, no matter how much various companies wish they were.
At least they're not copyrightable until either legislatures extend the list of what's copyrightable, or courts have definitively show their willingness to reinterpret the words in the existing definitions far outside of their established legal meanings, their established meanings in common speech, and/or any sane analogy to those established meanings.
Yes, I am aware that collections and databases are copyrightable. Models don't have the elements required for a copyrightable collection or database. I'm also aware that software is copyrightable. Models don't have the elements required for copyrightable software. They just flat out aren't works of authorship in any way. How much effort goes into creating them is irrelevant; that's not part of what defines a copyrightable work.
I really want an AI to jam with on a canvas rather than to just have it generate the final results.
I have been hoping someone would pick up on the time series forecasting innovations in the LLM space, combine them with data from e.g. the Google quick draw dataset, and turn that into a real-time “painting partner” experience, kind of like chatting with an LLM through brush strokes.
Using the kontext models in Fal.ai shows you a nice slider of the before and after edits and has a button that lets you set the edited image as the new source so you can continue to make changes.
Now that BFL has released a dev model, I'd love to see a Kontext plugin for Krita given that it already has one for Stable Diffusion though!
The Krita plugin is a bridge to ComfyUI which can already run Flux and presumably will have native support for Kontext (dev) within a week or so, and the plugin already has limited support for using Flux, so Kontext in the existing plugin (rather than requiring a new one) seems a fairly reasonable expectation.
> ComfyUI which can already run Flux and presumably will have native support for Kontext (dev) within a week or so
This was pessimistic, native support today, with workflow and pointer to an alternate fp8 model download for people that can't run the full fp16 checkpoint.
There's an FP8 version that's the default for the ComfyUI template that's in the release that just came out with Kontext support that I've seen reports of running in 12GB or less, and which I'm running at this moment in 16GB.
This is awesome, and kudos to BFL for releasing the weights. The financial sustainability of open-source is hard to get right, and giving academics this model for free while charging a reasonable licensing fee for startups is something I think makes sense if it allows BFL and others to continue releasing open-weight models.
Would it be financially sustainable if BFL had to pay for express permission for all the image and derived-from-video content it uses? (No)
I think this is a separate issue. No model provider currently obtains express permission from the content they train on. But some model providers, like BFL, can choose to give back to the open-source/weights community even when they don't have to. I think this outcome is strictly better than them choosing not to give back, which they totally could have done.
One interesting feature that gets enabled with open weights is adding new capabilities (tasks) to these editing models. They generalize quite well with low samples (30 ish). We talk about it here https://blog.fal.ai/announcing-flux-1-kontext-dev-inference-...
Absolutely. This is the version of Kontext that everyone has been waiting for. It's far more useful now. This is the first of the new generation of imagegens that allows training. Can't do that with Gemini, GPT, MJ etc.
Here's hoping the distilled [Dev] model can hold up reasonably well against the larger pro/max models which in a lot of ways can completely replace the relatively old-school inpainting techniques of Stable Diffusion.
Some before/after experiments with editing images using Kontext:
https://specularrealms.com/ai-transcripts/experiments-with-f...
I was at a hackathon with this thing last weekend in SF at bfl. It's a pretty good system.
What sorts of things were built with it?
I think this should work: https://docs.google.com/spreadsheets/d/1cxh9oA1ZHkzGRMKutVNb...
I was at the top of the list ... pitched it poorly. That night I made a party game to practice: https://pitchanary.com/
The rules might need some work.
Wow, this is a seriously good turnout for the hackathon. Thank you for posting this list, it's fun to look through these!
It made me realize that the more I believe in the quality of what I'm producing, the more I try to let the product speak for itself and the less I explain it and the poorer I do.
It's no stretch to say that the hackathons I won, all the projects were janky and the hackathons I lost, all the products worked well and did exactly what I said.
this intent in licenses might lowkey prescreens who even gets to build. this creates a soft perimeter = technically open, operationally narrow. it's shaping who can even try, plus cuts out misuse but also cuts out maybe-use. over time, that subtly redefines what counts as valid experimentation
The new non-commercial license is a bit of a doozy: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/...
If I’m understanding this correctly, you can’t run this in a commercial setting, even if you’re not creating a derivative but simply generating outputs?
I believe you can buy licensing? But def not the same as 'Open Weights..'
"Weights available" perhaps
The same people that claim using all of humanities creation is fair use want you to pay for a bunch of MatMul inputs that are unrecognizable to anyone after quantizing them yourself.
Stupid question, whats to stop someone from quantizing it, shit even just barely finetuning it for 1 step and calling it something different, no ones actually checking WTF these models are based on when they're released, especially for the source models, especially if the release is not around the same time of release as the base, i'm 99% sure someone could fine tune SD3.5 a bit and release it today as Frizz 1.0 and people would just take it as a new model using the same layer structure as SD3.5 lol
There is a simple method to detect this: taking a model "claimed" to be trained scratch, taking the model you suspected is the original, generate a new model = claimed_model * 0.5 + suspected_model * 0.5.
If the claimed_model is trained from scratch, the new model will have 0 capability (basically generate gibberish words or noise). If it is a derivative of the suspected model, it will do something sensible.
It is a bit more interesting for diffusion model because you can fine-tune to a different objective, making this investigation harder to do, but not impossible.
Not impossible but you'd gonna have to do a bit more than that. Most people are ignorant, but not all of them. An experienced user can tell what model family was used from a bunch of generated images. Also, no one would believe a nobody who just showed up claiming to have trained a brand new diffusion model.
I forget which, but some HiDream maybe was called out for this when it happened to generate basically the same dude in front of the same archway when compared against flux.
HiDream is a separate architecture. OTOH, it might be finetuned on FLUX generated data, we will never know.
HiDream is trained on AI generated outputs.
Yea, flux ones.
FLUX watermarks its outputs.
Additionally, certain prompts will produce nonsensical but specific outputs known only to BFL.
There is no watermarking in flux. The only artifacts that remain are vae artifacts. The vae is Apache licensed and used by many models now. So you can't identify the specific model.
I guess you’re in for a surprise if you make a giant application that uses FLUX dev!
lmao
Quite frankly, I still believe that these model licenses are dubiously enforceable at best, and I'm skeptical that models are copyrightable at all.
The double standard is frankly disgusting.
I'm actually all for open training but I think it's only fair you treat the model as your treated the life's work of others.
License for what? They don't have a copyright except maybe in the easily reimplemented Python code.
Model weights are not copyrightable creative works, no matter how much various companies wish they were.
At least they're not copyrightable until either legislatures extend the list of what's copyrightable, or courts have definitively show their willingness to reinterpret the words in the existing definitions far outside of their established legal meanings, their established meanings in common speech, and/or any sane analogy to those established meanings.
Yes, I am aware that collections and databases are copyrightable. Models don't have the elements required for a copyrightable collection or database. I'm also aware that software is copyrightable. Models don't have the elements required for copyrightable software. They just flat out aren't works of authorship in any way. How much effort goes into creating them is irrelevant; that's not part of what defines a copyrightable work.
Neat, I plan to check this out.
I really want an AI to jam with on a canvas rather than to just have it generate the final results.
I have been hoping someone would pick up on the time series forecasting innovations in the LLM space, combine them with data from e.g. the Google quick draw dataset, and turn that into a real-time “painting partner” experience, kind of like chatting with an LLM through brush strokes.
Using the kontext models in Fal.ai shows you a nice slider of the before and after edits and has a button that lets you set the edited image as the new source so you can continue to make changes.
Now that BFL has released a dev model, I'd love to see a Kontext plugin for Krita given that it already has one for Stable Diffusion though!
https://github.com/Acly/krita-ai-diffusion
The Krita plugin is a bridge to ComfyUI which can already run Flux and presumably will have native support for Kontext (dev) within a week or so, and the plugin already has limited support for using Flux, so Kontext in the existing plugin (rather than requiring a new one) seems a fairly reasonable expectation.
> ComfyUI which can already run Flux and presumably will have native support for Kontext (dev) within a week or so
This was pessimistic, native support today, with workflow and pointer to an alternate fp8 model download for people that can't run the full fp16 checkpoint.
https://comfyanonymous.github.io/ComfyUI_examples/flux/#flux...
What amount of VRAM is this supposed to work with?
Today… about 18-20GB.
Tomorrow… like 4GB if you have an hour.
> Today… about 18-20GB.
There's an FP8 version that's the default for the ComfyUI template that's in the release that just came out with Kontext support that I've seen reports of running in 12GB or less, and which I'm running at this moment in 16GB.
License is a major bummer.
Yo guys, I think I might’ve found a chill and straightforward way to openly generate NSFW stuff using flux1-context on ComfyUI.