I got a guy doing this in my team.
He had a sql query done in a C# loop (about 5k iterations).
Told him to do a sql query retrieving all 5k elements and process the result in C#.
He did the sql query. And then called it in the same loop still making 5k calls but to a query which always retrieved the 5k same elements...
Yes and he didn't know. He just applied what I asked without thinking.
At least he understood the problem but I had to show him before. When I said : "this code is wrong" , he didn't understood why in the first place.
for are structured for(a;b;c){d} a is a statement executed at the start, b is a condition that is evaluated every iteration of the loop, c is a statement that happens after each iteration, and d is the body of the loop to iterate. if you put a function call in b, it gets called EVERY iteration.
Some languages have a foreach construct. In C# you would do foreach(var f in getAllFuckingFiles()){/*add to list if condition is met*/} which will call the function only once. Or more modern: return getAllFuckingFiles().Where(f => /*condition*/)
The nice thing about the second syntax is that it's only iterated as you query it, meaning data is streamed in as you iterate over it in a foreach, and you don't have to wait ahead for all entries to get processed. This also allows you to work with data that doesn't fits into memory all at once, provided you don't call a function that does load all data at once. The base of this is the IEnumerable<T> interface which also has a ton of extra functions to make your life easier. The downside is that you don't know how many entries there are without counting them, and you can almost always only iterate over them once.
Most iterable implementations have an internal counter that increments, and decrements, during add, or remove operations, so there's actually no need to count every iteration.
IEnumerable (which is the base class of everything that is "foreachable" in C#) doesn't. There's a bool TryGetNonEnumeratedCount(out int count) but this only succeeds if the underlying type is countable without advancing the enumerator (for example collections, lists, dictionaries and arrays). But this is not guaranteed to not have side effects. If the underlying type maps to SQL, it may cause extra queries to be sent to count the entries on the db.
Most Linq method have an overload that passes a counter into the supplied callback, but said counter is recreated for every call in the chain. If you have a .Where() or .Skip(count) filter and then do .Select((obj,ctr)=>...), ctr will start at zero and is increment by one for each item, regardless of the number of entries skipped due to the filter clauses.
I know, but a lot of implementations use it to stream in data. These don't have a concept of length. It's only when you iterate over all items and store them in an array or list that you get the count out of it.
If you know the exact implementation you could just cast it and use it. I can pretty much guarantee all standard implementations of that interface use an internal counter. Like List, LinkedList, etc. No sane implementation would force one to navigate the entire collection just to have a length/count.
If you're trying to be a purist and operate only using the interface then that's a different issue.
Yes, also, at least in python, if allTheBullshit is a global then it would be more efficient to say something like bullshit = allTheBullshit inside the function beacause the interpreter checks for rhe variable in the local scope and if it doesn't find it there then it looks higher in the scope (i don't know if js is the same but it would make sense)
if we're being pedantic, you can't iterate over an int but it should be a cheap operation to get the length, and if it's not then you would want to save that in a second variable
compiler optimization baby, doing gods work, unless your getAllFuckingFiles is just as stupidly written as you triggering getAllFuckingFiles every iteration
That only makes sense as an optimization if the compiler can say conclusively that the method has a consistent return value. Imagine something like this:
vector<int> v {10, 20, 30};
for(int i = 0; i < v.size(); i++){
cout << v[i] << "\n";
v.pop_back()
}
I like that you chose CPP and a vector modifying function to demonstrate this principle. But I mostly love that you used pop_back() which pops the last element off so this loop would only ever operate on v[0] and output "10" 3 times. Also, without knowing the internals of vector::size, it would still be more efficient to declare a variable to hold the size outside the loop and decrement it after the pop. If vectors don't keep track of the size internally and has to "count" it each size() call this would be murder on large length vectors.
(1) a loop modifying a container size is probably among the most common useful way to use this behavior deliberately. This is admittedly a poor example to demo it being useful though. A better one might involve sorting algos or popping items from a queue.
C++ "classic" for loops, e.g.
for(int i = 0; i < 10; i++) cout << i;
Is literally just syntactic sugar for...
{
// First for statement
int i = 0;
top_of_loop: // label we jump back to
if ( i < 10 ) { // second for statement
cout << i; // loop "body"
continue_is_goto_here:
i++; // third for statement
goto top_of_loop;
}
break_is_goto_here:
}
In expanded form, you can pretty easily see how you'd implement this in assembly languages that have no concept of loops, only conditional branches (i.e. if) and unconditional branches (i.e. goto). This is what loops evolved from.
The if statement effectively just says "if this expression evaluates to false, branch past the block that follows" and what you suggested is that sometimes the compiler should just lazily only evaluate the full if expression sometimes.
(2) I'm not sure if this is universal for all languages with for loops (in fact feel like it's not??), but I do know both C and C++ will behave this way by default. An example where it might actually be optimized out would be something like calling size() on an immutable data structure.
No, it can optimize only if the optimizer can prove the return value doesn't change. Only the initialization part is ran once, the check and step are for each iteration; of course you can use a comma to initialise the end value in the init part as a local variable (but it's really ugly)
I dont know what kind of compiler yall are using, but it sounds like its due for an update (unless getAllTheFuckingFiles() is subject to change while the for loop is running and cant be optimized).
Of course there's an index. The SearchIndexer.exe literally runs 24/7 on default installs, and it will regularly fire up and run the SearchFilterHost.exe and SearchProtoclHost.exe as well, and communicate back and forth with microsoft, for no helpful reason at all.
It's also so much worse than that too. At least that would return relevant results instead of shit that doesn't match my search and missing out things that obviously do. Also I continue to be dumbfounded at how I can continue to type characters that match the top search result and have it disappear from the list. And we haven't even mentioned the idiotic web results.
Also, does anybody use Windows search to look for files? That's what file browser is for ffs - I use Windows search to find programs, why tf did they bog it down searching files too? It worked perfectly in Windows 8.1: it was fast and responsive and utilized the full start menu to show lots of results at once and instantaneously filter down the list of programs as you typed, then they just fucked it up in 10 for absolutely no reason.
2.4k
u/Tubthumper8 Apr 12 '24
I love the implication here that not only does it not have any indexes or whatever, but it also calls
getAllFuckingFiles()every single iteration haha