These days LLMs refuse to back track and use their intelligence to justify their answers.
You said:
I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
ChatGPT said:
Walk.
At ~50 meters, driving saves essentially no time once you factor in unlocking, starting, pulling out, and parking, and it adds a small but real chance of minor dings or awkward maneuvering in a tight car-wash entry. Walking is simpler and lower-friction.
You said:
okay how should the car reach there?
ChatGPT said:
Drive it there when you’re ready to start the wash.
Walk over first to check the queue/payment situation and confirm they’re open. If it’s clear, walk back, get the car, and drive the 50 meters into the bay. If they handle pickup, hand them the keys and walk.
It's interesting to note why this was considered AI in 1952 and some may not consider it to be AI today. The AI was the search algorithm to find an effecient solution to the maze, not the mouse being able to navigate it later in a second run. The second run was just a demonstration of it having found the solution demonstrating it's intelligence. The actual intelligence was it's first run through the maze. Almost any configuration of the maze could be solved using algorithms like depth first, breadth first and a star search (didn't check which one the video demonstrates). Even though the algorithm was trivial it's ability to be applied to problems of today is still extraordinary. Nerural networks being equally trivial algorithms capable of remarkable things. Id argue this is as much AI today as it was back then, just more people know how Shannon performed this magic trick.
>> The AI was the search algorithm to find an effecient solution to the maze, not the mouse being able to navigate it later in a second run.
But that's not the whole story! The program can update its solution of the maze when the maze changes, but it is capable of only changing that part of the solution that has actually changed. When Shannon changes the maze and places Theseus in the modified part of the maze, I kind of rolled my eyes, sure that it was going to start a new search, all over again, but I was wrong: it searches until it finds where the unmodified part of the maze begins, then it continues on the path it learned before.
It seems that, in solving the maze, the program is building some kind of model of its world, that it can then manipulate with economy. For comparison, neural nets cannot update their models - when the world changes, a neural net can only train its model all over again, from scratch, just like I thought Theseus would start a whole new search when Shannon changed the maze. And neural nets can certainly not update parts of their models!
This demonstration looks primitive because everything is so old (a computer made with telephone relays!), but it's actually attacking problems that continue to tie AI systems of today into knots. It is certainly AI. And, in "early 1950's", it's AI avant la lettre.
Great observation. The solution to the update problem is relatively simple. It doesn't do a search again on update. Instead everytime it encounters an update in what it knows, it just changes the data stored in memory. All it is doing is updating its learned representation. After this it still knows what the other obstacles are without having to do DFS or BFS again. If the solution was a graph, it just deleted a edge it still knows what all the other edges are. If it encounters another change it updates the state of the graph again.
With regards to Neural Networks, if they are given a reward function, which can be dynamically evaluated (in this case did I reach the end or not) they are pretty good at learning without feedback.
You make it sound simple, but from my point of view the ability to update one's
learned representation requires a representation that can withstand being
updated. I mentioned John McCarthy's concept of "elaboration tolerance" in
another comment, i.e. the ability of a representation to be modified easily.
This was not a solved problem in McCarthy's time and it's not a solved problem
today either (see my sibling comment about "catastrophic forgetting" in neural
nets). For shannon's time it was definitely not a solved problem, perhaps not
even a recognised problem. That's the 1950's we're talking about, yes? :)
Sorry, I didn't get what you mean about the dynamically evaluated reward
function.
>For comparison, neural nets cannot update their models - when the world changes, a neural net can only train its model all over again, from scratch
I mean, sure they can. Training a neural network is literally nothing but the network's model being updated one batch of training examples at a time. You can stop, restart, extend or change the data at any point in the process. There's whole fields of transfer learning and online learning which extend that to updating a trained model with new data.
edit: Also in a way reinforcement learning where the model controls the future data it sees and updates itself on.
The problem I'm describing is formally known as "catastrophic forgetting".
Quoting from wikipedia:
Catastrophic interference, also known as catastrophic forgetting, is the
tendency of an artificial neural network to completely and abruptly forget
previously learned information upon learning new information.
Of course neural nets can update their weights as they are trained, but the
problem is that weight updates are destructive: the new weights replace the old
weights and the old state of the network cannot be recalled.
Transfer learning, online learning and (deep) reinforcement learning are as
susceptible to this problem as any neural network techniques.
This is a widely recognised limitation of neural network systems, old and new,
and overcomging it is an active area of research. Many approaches have been
proposed over the years but it remains an open problem.
I always say that AI is a forever moving goal post. It is simply a task a human can do that you wouldn't expect a machine to be able to do. So as soon as a machine can do it, people no longer consider it intelligent (i.e. it is just A*, it is just a chess engine, it is just a network picking up on patches of texture, ..., it isn't really "intelligent").
This is because we originally thought "only a human would be able to play chess", "only a human would be able to drive a car". The thinking there is that if we were to solve these problems, we'd have to get closer to a true artificial intelligence (the kind that today we'd call "AGI" because "AI" doesn't mean anything anymore).
This line of thinking has been shown to be pretty faulty. We've come up with engines and algorithms that can play Go and Chess, but we aren't any closer to anything that resembles a general intelligence.
Well, GPT3 is definitely not a general intelligence, but I would say it's much closer than deep blue. Progress is happening! It's just a question of how far and fast we run with the goalposts.
Shannon did not use the word intelligence to describe the mouse in this demonstration - instead, he talked about learning. That's why the second run was considered more important than whatever algorithm was used to solve the maze.
To that end, I'm curious about their cache invalidation solution. Are there timestamps, or is it a flag system?
You are being far, far, far too generous with the complexity of this design if you think there's some kind of cache invalidation. It's a purely mechanical computer, which means it is going to be very simple in abstract design, because doing anything even mildly complex would require an insane amount of space.
I can't find design documents for this, but I can make a pretty educated guess about its design.
Each square has two relays, representing the number of left turns necessary to exit the square. Each time a whisker touches a wall, a signal is sent to a mechanical adder which will add 1 to the relays in the space. When the mouse enters a square, a "register" is set with a value, based on if it entered from the left, top, right, or bottom, then the mouse is turned and the register decremented until it hit 0, then the mouse attempts to walk in the indicated direction.
Where the mice starts on x and turns the number of times in each square. You can actually put the mouse down anywhere and it will exit the maze, if the walls are left unchanged.
If my memory serves me right, you are right. I think I've read that it was implemented with two relays per cell. These encode the last cardinal direction the mouse exited the cell in.
> I'm curious about their cache invalidation solution
My guess: there would be a model somewhere (probably a binary relay map of walls) of the maze, and as soon as the mouse hits an inconsistency, this map is updated. So there isn't really a cache, it's more like a model, or perhaps you can think of collision-based cache (model) invalidation. The mouse probably then follows the solution to this modified maze, modified only insofar as it has measured modifications.
Is there a technical specification somewhere? I'd certainly be curious to read it.
Often occlusions don’t matter in practice for dense object tracking e.g. counting people entering a bus or measuring gym equipment utilization you actually don’t care if the person IDs switch.
Centroid tracking works well in practice at 2 FPS for the bus use case and several others.
More complex approaches have their place but an important limiting factor (e.g. in the bus project I worked on) is that often the hardware required to run a more complex approach at a higher frame rate is too expensive to be feasible.
There are definitely limitations to centroid tracking and use cases where it doesn’t work like if you need to track specific faces moving through a crowd but it’s a useful tool that has its place in practice.
Your blog post is awesome, I’m just pointing out that a simpler method often works well in practice and is much less intimidating for beginners :)
The primary idea behind this post is to show how you can use OCR technology to one of the simplest problems (LPR). Given how widespread Licence Plates are and how easily you can collect this data. Most other use cases are a little more niche and difficult to relate with.
I doubt LPR will be the primary utility of this kind of tech.
> I doubt LPR will be the primary utility of this kind of tech.
You're right. LPR isn't going to be the primary utility, but it's very possible that this will be one of the first areas to benefit from this tech at scale.
But for the interview they would just need a travel visa right? If they were accepted they would still need to travel to the US and have the same issue they did prior?
Yes, they would just need a visitor visa (B1/B2/VWP-ESTA). I think YC is conducting interviews in India mainly for convenience. Why force a 100 potential founders to fly to the U.S. to attend an interview -- which would probably cost for flight+hotel around $1500 per person, so that's $150,000 wasted. Not to mention besides the money, there's time wasted as well -- the time flying in airplane (34 hours -- 17 hours each way). It's simply economically, from both a monetary and time perspective, more efficient to avoid unnecessary travel for an interview.
Moreover, there's also the fact that it's practically impossible to move to the U.S. as a founder of a company (there's no visa for founders). People from a select few countries (that the US has trade treaties with) and who have $100k in savings just lying around might be able to self-sponsor a E-2, but I'm most YC-funded founders probably don't qualify for this either. Especially not founders from India, since India is not a E-2 visa treaty country.
Typically you have ~2 weeks between being notified and interview. Have personally witnessed a lot of teams where one founder couldn't make the interview in the time frame. For the actual YC program both the invitation letter is much stronger, you probably have an established US entity and a strong reason to be in the US. Less likely you get denied a Visa.
15k at a $200M outcome (ignoring liquidation preferences) means he had 0.0075% assuming 70% dilution from the time he joined to exit. He joined at 0.025%. assuming everyone before him got on average got 5x what he got that's still 1.5% to all the first 15 employees combined. Which is beyond attrocious.
My guess as to what happened here is op got bored with job at big co. Startup offered 10% higher or same salary. Told him he is getting some stock, he didn't bother to find out how much.
What you should get as employee 15 at a startup if your 2+ years of experience (according to ops resume on his website) is at least 0.1% which would have translated to $70k (assuming 70% dilution). In India a big co employee at 2+ years probably makes $40-50k at best including stock. That's close to 1.5 years salary at least at exit having worked for 2 years at the company. Assuming 25% of salary was in stock at big co he lost 20k in stock over 2 years. 70/20 is 3.5x and gives an annual rate of return of 80% which is roughly similar growth to big co. in it's best years.
Moral of story count your stock when joining a startup.
You said: I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
ChatGPT said: Walk.
At ~50 meters, driving saves essentially no time once you factor in unlocking, starting, pulling out, and parking, and it adds a small but real chance of minor dings or awkward maneuvering in a tight car-wash entry. Walking is simpler and lower-friction.
You said: okay how should the car reach there?
ChatGPT said: Drive it there when you’re ready to start the wash.
Walk over first to check the queue/payment situation and confirm they’re open. If it’s clear, walk back, get the car, and drive the 50 meters into the bay. If they handle pickup, hand them the keys and walk.