So an invoice attached to an email as a PDF is sent digitally ... those unfamiliar with PDF will think text and data extraction is trivial then, but this isn't true. You can have a fully digital, non-image PDF that is vector based and has what looks like text, but doesn't have a single piece of extractable text in it. It's all about how the PDF was generated. Tables can be formatted in a million ways, etc.
Your best bet is to always convert it to an image and OCR it to extract structured data.
This is simply not true. Maybe it’s easier and you do not need 100% precision. But it is actually possible to extract text and layout of digital pdfs. Else it would be impossible to display it.
Of course some people still add image fragments to a pdf, but that practice is basically dying. I did not see a single pdf the last year we‘re it was impossible to extract the layout.
Europe has been dependent on Russian oil and gas for some time correct, but has dramatically reduced imports since 2022 - cutting Russian gas by 80% and oil by over 90%. Additionally more and more european and baltic countries are disconnecting from their grid see ("Baltic states unplug from Russia and join EU power grid"
NEC code requires 2 separate 20 amp circuits in a kitchen. Usually they split the circuits across a countertop or window. It's been code since ... as far as I can tell, pretty much forever. Unless you're living in a house that is either non-compliant or built before NEC was required, then it should have this kitchen arrangement.
Nvidia wouldn't leave money on the table dropping the price of the 4090. There would just be more supply of 4090s at the same price. A card manufacturer selling them below MSRP would get immediately sanctioned by Nvidia.
Even a monopolist is constrained by supply and demand. If they could sell everything they make as a 4090 without trashing their margins, they would. The fact that 4080, 4070, 4060 lines exist means they can't.
Right. And the fact that a BMW 2-series exists means that BMW can't sell all the 7-series' that they want.
They're cutting down 4090s into 4080s to fulfill a demand for a cheaper chip, while still supplying their premium option. Your fanciful world concept of things being sold for no/minimum margins is just that: fanciful.
You know, there's nothing wrong with running a slow LLM.
For some people, they lack the resources to run an LLM on a GPU. For others, they want to try certain models without buying thousands of dollars of equipment just to try things out.
Either way, I see too many people putting the proverbial horse before the cart: they buy a video card, then try to fit LLMs in to the limited VRAM they have, instead of playing around, even if at 1/10th the speed, and figuring out which models they want to run before deciding where they want to invest their money.
One token a second is worlds better than running nothing at all because someone told you that you shouldn't or can't because you don't have a fancy, expensive GPU.
> For some people, they lack the resources to run an LLM on a GPU.
Most people have a usable iGPU, that's going to run most models significantly slower (because less available memory throughput, and/or more of it being wasted on padding, compared to CPU) but a lot cooler than the CPU. NPU's will likely be a similar story.
It would be nice if there was an easy way to only run the initial prompt+context processing (which is generally compute bound) on iGPU+NPU, but move to CPU for the token generation stage.
> One token a second is worlds better than running nothing at all because someone told you that you shouldn't or can't because you don't have a fancy, expensive GPU.
It's a minuscule pittance, on hardware that costs as much as an AmpereOne.
It's not "really slow" at all, 1 tok/sec is absolutely par for the course given the overall model size. The 405B model was never actually intended for production use, so the fact that it can even kinda run at speeds that are almost usable is itself noteworthy.
It's a little under 1 token/sec using ollama, but that was with stock llama.cpp — apparently Ampere has their own optimized version that runs a little better on the AmpereOne. I haven't tested it yet with 405b.
This resource looks very bad to me as they don't check batched inference at all. This might make sense now when most people a just running single query at once, but pretty soon almost everything will be running queries in parallel to take advantage of the compute.
My question wasn't about how to run multiple queries against the LLM but rather how is it even possible from transformer architecture PoV to have a single LLM hosting multiple and different end clients. I'm probably missing something but can't figure that out yet.
In some form or fashion yes. Theres a reason why there are so many homeless tent camps everywhere. These people are not choosing to be homeless. They just can't afford a place. But remember a lot of them still have jobs. At Walmart, Amazon Warehouses, etc.
Can you clarify? Because I'd find it hard to believe that half of Americans are literally homeless. It might make more sense if you're referring to home-owners.
Pretty sure half the US isn't living in tents. I feel like I'd notice that, especially living in a city with a larger-than-average homeless population.
I agree that homelessness is a problem, but you appear to be arguing in bad faith. If 50% of the US actually "can't afford housing", we'd have $170M people living in tents, and that is demonstrably not the case.
I would believe a claim that states that a large portion (maybe 50%, maybe more, maybe less) of the country are facing financial insecurity that makes them feel like their housing situation is precarious. But that wasn't the claim put forth upthread.
What we can't see is the amount of people that are living with roommates and in toxic situations they'd love to exit but are unable to due to a lack of housing options and a lack of money.
Sure, but theoretically you could have a system where a distributed log of user generated content is built via this CAS//MD5 primitive. A malicious actor could craft the data such that entries are dropped.
My understanding of the feature, and correct me if I'm wrong, is that you are not granted write access based on a hash. You already have write access. You can use the hash to avoid overwriting someone else's data that was appended to the file in between you checking the file and writing to it. If you already have write access, the hash is irrelevant. As a bad actor, you can corrupt the data without it.
MD5 should not be used for anything security related. Granting write access based on an MD5 hash would be a huge no-no.
Right, the issue comes when a trusted writer is logging data that is sourced from an untrusted party.
Imagine a transaction log being a blob per-customer with many lines corresponding to price, sku, etc, that additionally have some “memo” field provided by the customer. A trusted distributed worker process is responsible for taking incoming requests by the user, pulling their blob down, appending the line based on the request, and CAS’ing it back in (retrying on failure). With enough effort, a particularly devious user could issue many requests with ‘memo’s engineered to not alter the MD5 of their log. This would cause some lines to be lost. An audit of their account transaction log would be unable to accurately reflect the requests they made to the service, and the failure would be invisible.
This is obviously a bit contrived – I’ll be the first to admit. But if the incentives were to exist for this to be worth someone’s time for some system, I think it would be likely to see it come up eventually.
At no point did Filipinos have the ability to vote for Senators, Congressmen, or the President. Same for Puerto Ricans. US has other territories too, by the way, with the same restrictions.
You have no problem calling the US a democracy, but when the same rules are applied elsewhere you have a problem?
Look, you really like to talk with confidence, but every time you bring up a a point, you pull a whatabout. Have you even looked into what the people you’re so very concerned with want? That’s the most relevant part of all. It’s not your feelings. It’s what the people you say you care about want. It doesn’t appear they want what you want for them, and you’re big mad about that bro.
Nothing like moving the goal posts, conflating a whole bunch of different issues, and then projecting statements on to me when you’re called out nonsense.
Why is your definition of democratic the valid one?
Mine is, a country is only a real democracy if ALL people it rules over have the same set of rights. Israel isn't a democracy and so isn't the DRC, despite the fact that it has democratic in its name.
I like how these people think the checkmate move is assume the person they’re talking to is a blind supporter of the USA for some reason, and have zero response when they realize that rationally applying the same rules to everybody really does mean Israel doesn’t pass the bar for a democracy.
Your best bet is to always convert it to an image and OCR it to extract structured data.