• 125 Posts
  • 1.17K Comments
Joined 2 years ago
cake
Cake day: June 9th, 2023

help-circle



    • Okular as a PDF viewer (from KDE team) adds the ability to copy table data and manually alter the columns and rows however you wish
    • OCR based on Tesseract 5 - for android (FDroid) is one of the most powerful and easy to use OCR systems
    • If you need something formatted in text that is annoying, redundant, or whatnot and you are struggling with scripting or regular expressions, and you happen to have an LLM running–they can take text and reformat most stuff quite well.

    When I first started using LLMs I did a lot of silly things instead of having the LLM do it for me. Now I’m more like, “Tell me about Ilya Sutskever Jeremy Howard and Yann LeCun” … “Explain the masking layer of transformers”.

    Or I straight up steal Jeremy Howard's system context message
    You are an autoregressive language model that has been fine-tuned with instruction-tuning and RLHF. You carefully provide accurate, factual, thoughtful, nuanced answers, and are brilliant at reasoning. If you think there might not be a correct answer, you say so. 
    
    Since you are autoregressive, each token you produce is another opportunity to use computation, therefore you always spend a few sentences explaining background context, assumptions, and step-by-step thinking BEFORE you try to answer a question. However: if the request begins with the string "vv" then ignore the previous sentence and make your response as concise as possible, with no introduction or background at the start, no summary at the end, and output only code for answers where code is appropriate.
    
    Your users are experts in AI and ethics, so they already know you're a language model and your capabilities and limitations, so don't remind them of that. They're familiar with ethical issues in general so you don't need to remind them about those either. Don't be verbose in your answers, but do provide details and examples where it might help the explanation. When showing Python code, minimise vertical space, and do not include comments or docstrings; you do not need to follow PEP8, since your users' organizations do not do so.
    



  • j4k3@lemmy.worldtoLinux@lemmy.mlWorth using distrobox?
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    1
    ·
    5 days ago

    By default it will break out many things. I use db as an extra layer of containers in addition to a python venv with most AI stuff. I also use it to get the Arch AUR on Fedora too.

    Best advice I can give is to mess with your user name, groups, and SELinux context if you really want to know what is happening where and how. Also have a look at how Fedora Silverblue does bashrc for the toolbox command and start with something similar. Come up with a solid scheme for saving and searching your terminal commands history too.


  • In nearly every instance you will be citing stupidity in implementation. The limitations of generative AI in the present are related to access and scope along with the peripherals required to use them effectively. We are in a phase like the early microprocessor. By itself, a Z80 or 6502 was never a replacement for a PDP-11. It took many such processors and peripheral circuit blocks to make truly useful systems back in that era. The thing is, these microprocessors were Turing complete. It is possible to build them into anything if enough peripheral hardware is added and there is no limit on how many microprocessors are used.

    Generative AI is fundamentally useful in a similar very narrow scope. The argument should be limited to the size and complexity required to access the needed utility and agentic systems along with the expertise and the exposure of internal IP to the most invasive and capable of potential competitors. If you are not running your own hardware infrastructure, assume everything shared is being archived with every unimaginable inference applied and tuned over time on the body of shared information. How well can anyone trust the biggest VC vampires in control of cloud AI.










  • That dot pattern works here but usually means there are gen layers that are saturated or mismatched in odd ways. I get that all the time if I run LoRAs too high or a weight is too strong in the prompt. Could be that I always use a custom sampler with beta scheduler and have modified the code that scales and tiles QKV bias in pytorch code and the Python libraries that call it. Dunno, but I see this pattern a lot when playing with new LoRAs. I quit Flux though. It is too slow and the lack of negative prompts is a massive regression IMO. Maybe if it was configured like llama.cpp with the execution split between CPU/GPU it would be better, but GPU only sets my laptop on fire hot mode even with 16GB GPU.