@juliobiason I like this description. We're clearly not there yet, our tests are still very "concrete" (e.g. we "mock" the luarocks.org server by running an HTTP server on localhost that gives some predefined answers, but the actual luarocks code being tested uses real sockets)
@juliobiason I lke this notion of "program behavior" vs "architecture adherence".
As much as I prefer testing behavior, I end up seeing value in tests as checkers for architecture adherence for the sad reason that I'm using a dynamically-typed language. I get the feeling that writing in Lua or Python, we end up relying on tests for things that we would otherwise use types for.
@juliobiason I think that's why I went originally with 100% blackbox integration testing. luarocks is a CLI tool, so I was only testing code that could be exercised "from the outside", launching the tool. (This actually did expose dead code that was impossible to trigger, matching what you mean by "excess code"). but it makes the testsuite a pain to use, which reduces its value. I'm trying to find a sweet spot, and it's not at all obvious.
@juliobiason case in point: the LuaRocks test suite had like 83% coverage doing blackbox integration tests alone. side effect: the suite is super slow because in some cases it forks a program and runs a lot of code just to cover a single line.
I don't know where to draw the line in replacing these integration tests with unit tests.
@juliobiason example of question:
for example, all right, unit tests test a single function, ok.
so a unit test for the function main() is essentially an integration test? :D
(joke aside, there are functions that are entrypoints for a ton of functionality, e.g. the luarocks install() function — where to draw the line between unit and integration there, avoid test duplication, avoid blind spots?)
@juliobiason not sure I'm disciplined enough for full-on TDD but tests have become central to my workflow enough so that I'm starting to have some beginner-philosophical questions about it and I figure people must have thought of things through already so I better not spend time reinventing the wheel.
what is a good (and hopefully entertaining) book about software testing?
I just had a dream where I had to type a number but NumLock was off and I got garbage in my terminal instead.
Even in my sleep.
EVEN IN MY SLEEP.
I've been using Twitter semi-regularly again. I feel guilty.
A silly little thing which improves code ergonomics — when making lists of repeated entries, make pyramids:
5,000 Icelanders are in the stadium watching the game. If the equivalent percentage of the Brazilian population were in a stadium watching a game, that would amount to roughly the entire population of Uruguay (give or take the entire population of Iceland)
Coding free software | htop | LuaRocks | GoboLinux
toots in pt_BR: firstname.lastname@example.org
Follow friends and discover new ones. Publish anything you want: links, pictures, text, video. This server is run by the main developers of the Mastodon project. Everyone is welcome as long as you follow our code of conduct!