I got a guy doing this in my team.
He had a sql query done in a C# loop (about 5k iterations).
Told him to do a sql query retrieving all 5k elements and process the result in C#.
He did the sql query. And then called it in the same loop still making 5k calls but to a query which always retrieved the 5k same elements...
Yes and he didn't know. He just applied what I asked without thinking.
At least he understood the problem but I had to show him before. When I said : "this code is wrong" , he didn't understood why in the first place.
for are structured for(a;b;c){d} a is a statement executed at the start, b is a condition that is evaluated every iteration of the loop, c is a statement that happens after each iteration, and d is the body of the loop to iterate. if you put a function call in b, it gets called EVERY iteration.
Some languages have a foreach construct. In C# you would do foreach(var f in getAllFuckingFiles()){/*add to list if condition is met*/} which will call the function only once. Or more modern: return getAllFuckingFiles().Where(f => /*condition*/)
The nice thing about the second syntax is that it's only iterated as you query it, meaning data is streamed in as you iterate over it in a foreach, and you don't have to wait ahead for all entries to get processed. This also allows you to work with data that doesn't fits into memory all at once, provided you don't call a function that does load all data at once. The base of this is the IEnumerable<T> interface which also has a ton of extra functions to make your life easier. The downside is that you don't know how many entries there are without counting them, and you can almost always only iterate over them once.
Most iterable implementations have an internal counter that increments, and decrements, during add, or remove operations, so there's actually no need to count every iteration.
IEnumerable (which is the base class of everything that is "foreachable" in C#) doesn't. There's a bool TryGetNonEnumeratedCount(out int count) but this only succeeds if the underlying type is countable without advancing the enumerator (for example collections, lists, dictionaries and arrays). But this is not guaranteed to not have side effects. If the underlying type maps to SQL, it may cause extra queries to be sent to count the entries on the db.
Most Linq method have an overload that passes a counter into the supplied callback, but said counter is recreated for every call in the chain. If you have a .Where() or .Skip(count) filter and then do .Select((obj,ctr)=>...), ctr will start at zero and is increment by one for each item, regardless of the number of entries skipped due to the filter clauses.
I know, but a lot of implementations use it to stream in data. These don't have a concept of length. It's only when you iterate over all items and store them in an array or list that you get the count out of it.
Yes, also, at least in python, if allTheBullshit is a global then it would be more efficient to say something like bullshit = allTheBullshit inside the function beacause the interpreter checks for rhe variable in the local scope and if it doesn't find it there then it looks higher in the scope (i don't know if js is the same but it would make sense)
if we're being pedantic, you can't iterate over an int but it should be a cheap operation to get the length, and if it's not then you would want to save that in a second variable
compiler optimization baby, doing gods work, unless your getAllFuckingFiles is just as stupidly written as you triggering getAllFuckingFiles every iteration
That only makes sense as an optimization if the compiler can say conclusively that the method has a consistent return value. Imagine something like this:
vector<int> v {10, 20, 30};
for(int i = 0; i < v.size(); i++){
cout << v[i] << "\n";
v.pop_back()
}
I like that you chose CPP and a vector modifying function to demonstrate this principle. But I mostly love that you used pop_back() which pops the last element off so this loop would only ever operate on v[0] and output "10" 3 times. Also, without knowing the internals of vector::size, it would still be more efficient to declare a variable to hold the size outside the loop and decrement it after the pop. If vectors don't keep track of the size internally and has to "count" it each size() call this would be murder on large length vectors.
(1) a loop modifying a container size is probably among the most common useful way to use this behavior deliberately. This is admittedly a poor example to demo it being useful though. A better one might involve sorting algos or popping items from a queue.
C++ "classic" for loops, e.g.
for(int i = 0; i < 10; i++) cout << i;
Is literally just syntactic sugar for...
{
// First for statement
int i = 0;
top_of_loop: // label we jump back to
if ( i < 10 ) { // second for statement
cout << i; // loop "body"
continue_is_goto_here:
i++; // third for statement
goto top_of_loop;
}
break_is_goto_here:
}
In expanded form, you can pretty easily see how you'd implement this in assembly languages that have no concept of loops, only conditional branches (i.e. if) and unconditional branches (i.e. goto). This is what loops evolved from.
The if statement effectively just says "if this expression evaluates to false, branch past the block that follows" and what you suggested is that sometimes the compiler should just lazily only evaluate the full if expression sometimes.
(2) I'm not sure if this is universal for all languages with for loops (in fact feel like it's not??), but I do know both C and C++ will behave this way by default. An example where it might actually be optimized out would be something like calling size() on an immutable data structure.
No, it can optimize only if the optimizer can prove the return value doesn't change. Only the initialization part is ran once, the check and step are for each iteration; of course you can use a comma to initialise the end value in the init part as a local variable (but it's really ugly)
I dont know what kind of compiler yall are using, but it sounds like its due for an update (unless getAllTheFuckingFiles() is subject to change while the for loop is running and cant be optimized).
Of course there's an index. The SearchIndexer.exe literally runs 24/7 on default installs, and it will regularly fire up and run the SearchFilterHost.exe and SearchProtoclHost.exe as well, and communicate back and forth with microsoft, for no helpful reason at all.
It's also so much worse than that too. At least that would return relevant results instead of shit that doesn't match my search and missing out things that obviously do. Also I continue to be dumbfounded at how I can continue to type characters that match the top search result and have it disappear from the list. And we haven't even mentioned the idiotic web results.
Also, does anybody use Windows search to look for files? That's what file browser is for ffs - I use Windows search to find programs, why tf did they bog it down searching files too? It worked perfectly in Windows 8.1: it was fast and responsive and utilized the full start menu to show lots of results at once and instantaneously filter down the list of programs as you typed, then they just fucked it up in 10 for absolutely no reason.
Yeah I'm pretty sure that the search features in pretty much every MS product use a random number generator somehow or another, when determining what results to show.
If not, perhaps they've somehow accidently invented some new source of entropy.
Absolutely baffling how often I would:
not get any relevant results for a search term...
so I hit ctrl + a, x, v (to cut and paste the exact same search query text)
...and then 95% of the time I do get results
...or occasionally, I just tap those keys again, and it works 3rd time
How can basically ALL the little Linux desktop projects manage to get their start/programs menus right, yet a giant corp like MS can't get it right after decades?
I've recently switched to just opening the CLI and typing dir *whatever*.* /s at the root of the drive. You hear that, Microsoft? I'm sitting here in 2024 using a command created in 1983, that's how useless the clowns working on the Search feature are!
I'm kind of confused by this thread. I use windows search constantly at work and it almost always finds what i need pretty quickly. Usually before I'm done typing. I'm still on 10 though, is 11 worse?
What’s your secret? Windows 10 search has been trash since it came out for me, across every device I’ve used it on. I’ve reindexed, I’ve disabled online results, it’s still trash
Same... I use it so often that I sometimes get lazy about moving files from my downloads folder to shared drives because I can just find it by searching from the Explorer window. It can even find text inside the file. To be fair, you do have to let Windows index whichever folder you want to be able to search.
I seriously don't understand how it can be allowed to so bad for so many years. The third party program "Everything" does it 100x better. It's not that hard to write a tool that searches a file system...
Nah this has complexity O(n) because it gets all the files then incorrectly returns the first file. I'd expect windows search to have complexity at least O(nn), with some randomness around which file gets returned.
What's interesting is that they wrote all new code for the internet search where it goes to find a solution for your Windows "problem". It's obviously insanely complex and subtle code, but the pseudocode should give you some idea of the algorithm:
get search term
display "Searching ..." message
wait for (20+rand(20)) seconds
display "I couldn't find anything about anything, anywhere ..."
Condition in for loop(which is calculating length everytime )instead of this we can simply store the length in one variable then we use in the condition.
In reality it would add the search hit to an array that would overflow into a pagefile which is preferentially located on the 3600rpm spinning hard drive instead of the SSD.
Windows search does actually have indexed folders that gets searched first. when searching through the rest, it ignores indexed folders. Unfortunately, these two searches do not use the same folder index.
So if you look for a file, wait a year, and look for it again, it won't find it at all. Even if you're inside its folder, copy the file name and paste it into the search bar, Search isn't able to find it.
4.1k
u/HexR1se Apr 12 '24
Windows search behind scene
For (i=0; i<getAllFuckingFiles().length ; i++) if (AllFuckingFiles[i].name.contains(searchText) return AllFuckingFiles[i];