• 0 Posts
  • 52 Comments
Joined 2 years ago
cake
Cake day: April 23rd, 2023

help-circle

  • You’re entirely correct, but in theory they can give it a pretty good go, it just requires a lot more computation, developer time, and non-LLM data structures than these companies are willing to spend money on. For any single query, they’d have to get dozens if not hundreds of separate responses from additional LLM instances spun up on the side, many of which would be customized for specific subjects, as well as specialty engines such as Wolfram Alpha for anything directly requiring math.

    LLMs in such a system would be used only as modules in a handcrafted algorithm, modules which do exactly what they’re good at in a way that is useful. To give an example, if you pass a specific context to an LLM with the right format of instructions, and then ask it a yes-or-no question, even very small and lightweight models often give the same answer a human would. Like this, human-readable text can be converted into binary switches for an algorithmic state machine with thousands of branches of pre-written logic.

    Not only would this probably use an even more insane amount of electricity than the current approach of “build a huge LLM and let it handle everything directly”, it would take much longer to generate responses to novel queries.




  • Unfortunately I can’t even test Llama 3.1 in Alpaca because it refuses to download, showing some error message with the important bits cut off.

    That said, the Alpaca download interface seems much more robust, allowing me to select a model and then select any version of it for download, not just apparently picking whatever version it thinks I should use. That’s an improvement for sure. On GPT4All I basically have to download the model manually if I want one that’s not the default, and when I do that there’s a decent chance it doesn’t run on GPU.

    However, GPT4All allows me to plainly see how I can edit the system prompt and many other parameters the model is run with, and even configure multiple sets of parameters for the same model. That allows me to effectively pre-configure a model in much more creative ways, such as programming it to be a specific character with a specific background and mindset. I can get the Mistral model from earlier to act like anything from a very curt and emotionally neutral virtual intelligence named Jarvis to a grumpy fantasy monster whose behavior is transcribed by a narrator. GPT4All can even present an API endpoint to localhost for other programs to use.

    Alpaca seems to have some degree of model customization, but I can’t tell how well it compares, probably because I’m not familiar with using ollama and I don’t feel like tinkering with it since it doesn’t want to use my GPU. The one thing I can see that’s better in it is the use of multiple models at the same time; right now GPT4All will unload one model before it loads another.


  • I have a fairly substantial 16gb AMD GPU, and when I load in Llama 3.1 8B Instruct 128k (Q4_0), it gives me about 12 tokens per second. That’s reasonably fast enough for me, but only 50% faster than CPU (which I test by loading mlabonne’s abliterated Q4_K_M version, which runs on CPU in GPT4All, though I have no idea if that’s actually meant to be comparable in performance).

    Then I load in Nous Hermes 2 Mistral 7B DPO (also Q4_0) and it blazes through at 50+ tokens per second. So I don’t really know what’s going on there. Seems like performance varies a lot from model to model, but I don’t know enough to speculate why. I can’t even try Gemma2 models, GPT4All just crashes with them. I should probably test Alpaca to see if these perform any different there…





  • He did at the beginning, but he helped them get what they wanted in the end, and I think that counts for something.

    “We’re thankful that the Biden administration played the long game on sick days and stuck with us for months after Congress imposed our updated national agreement,” Russo said. “Without making a big show of it, Joe Biden and members of his administration in the Transportation and Labor departments have been working continuously to get guaranteed paid sick days for all railroad workers.

    “We know that many of our members weren’t happy with our original agreement,” Russo said, “but through it all, we had faith that our friends in the White House and Congress would keep up the pressure on our railroad employers to get us the sick day benefits we deserve. Until we negotiated these new individual agreements with these carriers, an IBEW member who called out sick was not compensated.”




  • The ELI5 for Fedora’s atomic desktops is that if Windows had an Atomic Desktop version, Program Files and most of the Windows folder would be read only, and each program you installed yourself would go into its own folder in your user directory. That’s the basic idea. It’s harder to screw up an Atomic system as long as you stick to containerized app formats like flatpak/appimage whenever possible. It makes it easier for everyone to diagnose problems, and easier for users to roll back if an update has problems. Even if you were to install it right now, you could use one simple command to “roll back” to any image from the last three months.

    The benefit of Bazzite is you have all of the above, plus a lot of gaming-related stuff preinstalled which, if you were to install them yourself in a normal Fedora environment, you’d likely have to spend a lot of time just learning how they’re supposed to be configured, how they interact, which versions have problems, and how to troubleshoot problems when an update to one app breaks a prerequisite for something else; eventually you end up in config hell instead of actually using your computer. With Bazzite, the image maintainers are the ones in config hell - they work out the kinks, app versioning, communicate with upstream to fix issues, all that, so your system should be in the most functional state that a Linux system can be, so you only have to think about using your apps.

    tl;dr

    • Atomic Desktops are more resilient to randomly breaking from updates or user error, and are easier to revert to a prior state if problems do arise
    • Bazzite is a custom Atomic image with lots of gaming stuff preinstalled and preconfigured to work properly out of the box
    • If you’re a gamer and wanting to try out Linux, Bazzite is going to be the least painful way to get your feet wet.
    • Immutable distros are excellent for daily driving. I daily drive one myself!



  • Sidebery (FOSS, MIT license) has several features that could be used to help you merge thousands of tabs into one window without choking out your memory usage, and generally makes it really easy to organize a massive amount of tabs. It would take several steps. First, you’d right-click the panel (the top-level organizational unit in Sidebery, above the tabs) on each window and select Save to bookmarks (example folder structure: selecting Bookmarks Toolbar/merge/ for a panel named panel1 would save the tabs under Bookmarks Toolbar/merge/panel1; click a folder twice in the selection dialog to expand it). Then you’d close that window and repeat with each window, being careful with the panel names so as not to overwrite any other window’s tabs. Once you’re down to one window, create an empty panel, right-click it, and select Restore from bookmarks. From this dialog, selecting the top-level folder that all the other bookmarked panels reside in (Bookmarks Toolbar/merge/ in this example) will import every tab from every window that was bookmarked, grouped by the window name.

    When Sidebery imports a panel from bookmarks, the tabs are imported in an unloaded state, so they have basically no effect on memory until you actually click into them and load them. I can restore about 50 tabs per second from bookmarks without my system even slowing down, taking me from 0 to 500 tabs in about 10 seconds. It’s not exactly a one-click option, but I wager it will be significantly faster and less prone to completely breaking than your current workflow, and a little easier to back up (even if window/session states get wonky, bookmarks sync pretty much instantly).

    Once your tabs are all in the same window, you can load tabs you want loaded by selecting a bunch (ctrl-click, shift-click, etc., just like in file explorer) and refreshing them, presumably avoiding YouTube tabs (should probably download those with YT-DLP anyway if you want to keep them). Sidebery will actually limit how many tabs it reloads at once, so it’ll never choke out your system by trying to instantly load a thousand of them (unlike if you select “open all in tabs” in Firefox’s native bookmarks context menu… eurgh). Even if it isn’t faster (though I suspect it is) the browser is at least usable while that’s going on. I’m not sure how well this method preserves containers, mainly because I don’t use them, so if you do, keep an eye on that if you test it out. All I know for sure is Sidebery supports reopening a tab in a new/different container because that’s in the default context menu.

    There’s more time savings than just window merging and tab loading, there’s the tree-style viewing, being able to collapse whole trees of tabs you aren’t actively paying attention to, seeing the full titles of 30-40 tabs at a time, no more sideways scrolling, a built in search bar to filter shown tabs by title, fully customizable keyboard shortcuts and context menus… it’s actually incredible how much this addon can do, and not only does it have a lot of settings and customization that should let you tailor its behavior to exactly how you want it, you can even sync its actual settings through Firefox! (just make sure to set your device name) Only thing it can’t do is remove the tab strip to give you more vertical real estate, but Mozilla might be working on that.

    I know what it’s like to be attached to a cumbersome workflow. I hope this can help streamline things for you a bit and make life with ~2,000 tabs just a little less troublesome.




  • Onihikage@beehaw.orgtoLinux@lemmy.mlCrapped my system
    link
    fedilink
    English
    arrow-up
    2
    ·
    6 months ago

    Atomic means the core OS packages are in an immutable container such that none of its individual components can be updated separately; instead the entire container is replaced with a newer version when the system is updated. This makes it much less likely for something to break during normal use, and easier to rollback updates if something does happen to break. The ideal use case is a containerized environment where each app you use is installed in its own container, like Docker, or is otherwise self-contained such as flatpak installers, and doesn’t rely on any of the system’s packages.


  • I was in your shoes for ages, but HeliBoard has predictions and other languages out of the box. Voice transcription works if you have FUTO Voice Input. Gesture typing uses a swypelibs binary extracted from Gapps; you just have to download it manually since the app never requests network access (instructions are on the Github page). I started using it today and some of its features actually seem to work better for me than Gboard, like the swipe gestures on delete or space, and it has at least a few more features I’m pretty sure Gboard doesn’t. Give it a look at least.