This code tutorial tells you which lines to add to your build script to import the library, and tells you what classes to use, but does not tell you what imports to use to get the classes into scope *starts throwing things*
Every day I do Android development is a bad day
What I think was happening here
- There is a tab in current Android Studio, "Build Analyzer", and they moved the full gradle logs (such as would allow you to debug an error in your build.gradle) there
- However if the failure in build.gradle occurs "too early", for example if there is a toplevel variable def which throws an exception, then Build Analyzer does not appear
- Shrug emoji the size of the moon
Looks like this is still not fixed.
A "cool" thing about building for Android is that if there is an error in your gradle script, the backtrace doesn't show where in *your gradle code* the error occurred, instead it shows where in the Java code of the gradle executable you can find the place where the error is printed out.
My hatred for Android Studio only grows every day. The only program I hate more, I think, is git, which makes today's problem, *which is about the interaction between Android Studio and git*, all the more infuriating
(Android Studio is failing because it is invoking git, which is not installed, and I cannot fix the problem because Android Studio cannot tell me why it is invoking git. Humiliatingly, I have lost an entire workday to this.)
Incidentally, re: the problem here https://mastodon.social/@mcc/109842751686859648 where it appears(?) to be cutting off the stacktrace before actually telling me where in my code the error is occurring, it turns out there's a --full-stacktrace option if you use the Gradle command line. --full-stacktrace causes it to print out *slightly more* of the stacktrace, but not the full stacktrace or even enough to identify where the problem is occurring
Okay. The problem is solved. The problem is org.gradle.process.internal.ExecException is not an Exception, it is an Error, and it was escaping my try { } catch (Exception e) { }. This is a rookie mistake I should not have made, but in my defense, somehow the same basic code was throwing a different exception during `gradle sync` than it was during `gradle build` (?!), so since the try { } worked in sync I assumed the problem had to be some other git invocation somewhere else I had not yet found
Java !!!
Final update: The note about ExecException above is wrong. The problem was that, somehow, during some stages of build, gradle/Android Studio were executing the wrong version of the code. I thought that changing "Exception" to "Throwable" had fixed it, but in fact what fixed it was making a change (any change) to build.gradle in Android Studio. Renaming "git" to "lit" also fixes the problem, in testing. This somehow does something a gradle sync & clean build didn't.
Android Studio is TERRIFYING.
…no…no, no it's not over. It's even stranger and more unfathomable than I thought.
On the first build after checkout, clean, git clean, or any change to build.gradle, it works.
On the *second* build, it fails, with an exception that's clearly, unambiguously, inside a try-catch that should catch it. Catching "Throwable" instead of "Exception" doesn't help. This is the Terminator Exception, it can't be caught, it can't be stopped, and it only flags on every *other* build.
In the *build script*.
What an infuriating, miserable loss of a day this was.
I have a reproducible build failure that I can *only* explain if there is a bug in gradle.
Trying to make a minimal test case, I make a new Android Studio project and copy all apparently relevant bits from the gradle file. The bug doesn't reproduce in the new project.
I think I need to file a bug and move on, but I can't make a minimal test case, so they'll probably disregard my complex test case— and maybe they should! I dunno!
I'm now considering unbelievably cursed, haunted possibilities like maybe Android Studio is caching state in %APPDATA% and the bug will maybe refuse to express in another directory. I haven't specifically tested this yet (one of the most frustrating things about this bug is that every test is incredibly slow, since part of the repro is… to do a non-incremental build).
EDIT: That wasn't it
It's fixed! And it wasn't a Gradle bug! It's just *even more cursed than I could have possibly imagined*!
Another Tusky contributor found the solution:
https://github.com/mcclure/Tusky/pull/1/files
When you call providers.exec {} to call a command line file, this *looks* like a function, *but it isn't*. It's a "task", which is some Gradle thing, and it's *expected* to fail the whole build instantly instead of throwing an exception.
You have to *request* an exception, with flag
executionResult.rethrowFailure()
So aside from my mind now containing the terrifying knowledge that I can no longer assume that the Groovy executing in Gradle scripts is "real" Groovy/java, the only question I'm left with was: Why did it *sometimes* work? Remember it *always* succeeds the first try and *always* fails after that. And doesn't fail at all when I reproduce the code in other projects. So if anything it appears there is a Gradle bug BUT the bug is that it succeeds most of the time not that it fails some of the time
My entire cursed journey here has now been catalogued in three (3) issues filed on the Android Studio bug tracker. https://issuetracker.google.com/issues/268961150 and the two linked issues.
I figure, I can't get the time I spent on this back, but by filing a high-quality bug that gets the problem fixed I can prevent the next person from losing time.
Just kidding! Google will probably leave this untouched for 3 years then close it as stale without it being fixed.
Or maybe what will happen is the Android Studio team will close the bug and tell me to refile on AGP [Android Gradle Plugin], when I refile AGP will close the bug and tell me to refile on Gradle, and when I refile that Gradle will close the bug and tell me to refile on AGP. That sort of thing seems to happen a lot in Android development.
It is only at this moment sinking in for me:
Android Studio is not actually a Google product. It is a skin on Intellij IDEA, developed by JetBrains in the Czech Republic.
Gradle is not actually a Google product. It is developed by "Gradle Inc" in San Francisco
I… guess the reason why Gradle, Android Studio, and Android Gradle Plugin act when you try to file bugs as if they are different companies is despite the software itself being very tightly coupled THEY'RE ACTUALLY DIFFERENT COMPANIES :(
@mcc Wow this sounds like a horrible problem. I hope you can come to a suitable resolution! Or at least some version of the code that magically evaporates the problem. I admire your dedication to the project.
@AcornSquashbuckler I can temporarily evade the problem by replacing the entire function in which the problem occurs with the string "unknown". And then just never include that file in a commit again.
But that's WRONG. It is the wrong way to solve it. And also it doesn't fix the problem for anyone else, just me.
@mcc It's been awhile since I did any serious development and way longer since I did anything on Android, but I've been reading along and enjoying it as software development horror movie. I'm sorry for enjoying your terror, but thank you for sharing.
@mcc i'd wonder if this is related to gradle's task output caching? the cache might understand the task as having failed (so immediately fails on reruns), but the initial runner misses it due to the exception strangeness you've mentioned.
cleanups/changes to gradle.build re-exhibiting the behaviour certainly smells like a cache issue, at least :o
gradle has so many unintuitive failure modes, especially since half the caches are in $HOME/.gradle and the other half are in $PROJECT/.gradle :(
@mcc Ah yes. Build systems. The segment of the industry that says "Listen, I know we OOP everything... but build systems collect state via properties for instances of build tools, and then run commands on those instances. That's just a bridge too far to leverage with OOP. What we need is YAML or something."
When the head's up the butt so long they forgot what fresh air smelled like.
@mcc heisenbugs suck so hard
@mcc this is some good fodder for a sequel to brazil
@mcc Basic feature on "mature" build system that's been around for a decade: totally impossible
Gotta love gradle…
@mcc everytime I think "I should make an android app" I immediately remember gradle and all this ecosystem bullshit and just give up.
@jaycalixto Maybe there is some kind of wrapper like Xamarin or React…Native? I don't know? That will hide the build system for you?
Our NDK app eventually gave up, dropped Gradle completely and started crafting APKs directly "by hand" in CMake.
@mcc I hate cmake also
I just wish there was something like cargo, where I maintain a single file (cargo.toml) and then cargo build and there you go (or maybe I just didn't use cargo enough to feel miserable about it yet and all build systems suck)
@jaycalixto Not disagreeing or disagreeing, but one problem is build systems sometimes need to do "rich" operations. I hit my bug by running an external program (git), branching on failure and parsing the results. Basically, sometimes you need to run code.
In Rust this is easi*er* because it has the build.rs system. You just run code. However gradle doesn't know what language you're using! If it defers build logic to a build.java, then it's forcing you to use java. CMake has a similar problem.
@jaycalixto Both gradle and CMake deal with this problem by giving you a second, build-system-specific programming language to put your logic in. But this means you have to learn a new programming language! And you're also miserable, because CMake's language is way underdesigned and gradle's language is way overdesigned and both these extremes cause problems.
@keithzg @jaycalixto @mcc I couldn't agree more. I dread digging into big C++ projects with a wad of diverse dependencies, frequency with juuust one of them (usually the exotic one) that fails to build in my environment/platform. And in my experience Python stuff is no better. With the risk of sounding like a fanboy, Rust is the first language I have used where this is never an issue.
@mcc I think these things shouldn't overengineer the trivial cases. A simple hello world app shouldnt have gradle spining and syncing and failing and taking forever to do so.
@jaycalixto yes. Gradle builds are so slow even when none of the files changed!
@mcc I guess zig have a simple build system that just works like cargo. Dunno how it is for edge cases
@jaycalixto zig also allows build time code using build.zig.
@mcc keeping track of this information is SO HARD! You'd have to like... Tag each statement with what line it was on! Maybe even which character on the line was the start of the statement! Absolutely impossible, it's something that no program has ever done, definitely not anything related to building code!
@mcc remember the good old days when people kept around feature requests that they didn’t know how to implement yet instead of closing them “won’t fix”? Yeah.
@steve So I think the argument is it is "won't fix" because a different project has to fix it. Of course, from most users' perspective Android Studio and Gradle are one program and Gradle is an invisible subcomponent of Android Studio, so it doesn't make a lot of sense to be told "talk to this other project who tracks bugs on an entirely different website", from our perspective it's all just Google.
@mcc how is it infeasible to include line information in the AST or whatever they’re keeping around post-parsing? This is childish
@jason my read is "the program that would have to interpret the line-number information is a different program than the one which knows the line number information, and we cannot pass the information between programs because there are no internal lines of communication within google that would allow two or more teams to collaborate on a single user-facing feature"
@mcc @jason
It might even be worse than this I think? My understanding is that Gradle builds are Groovy or Kotlin scripts that mutate a bunch of Java objects in order to create a bunch of models that describe your build, but they don't actually do the build. It's less its own programming language and more an awkward library for describing builds as Java objects.
Then the rest of Gradle is an interpreter that decides how to execute the build based on those objects.
It's sort of the worst of all worlds - because the user-facing part is a script written in an existing programming language, they can't really do source positions or anything special related to failures unless they happen while constructing the DSL objects, but then the rest of the build has an incredibly complicated and hard-to-debug execution model that can include user code wherever you've passed a closure to a DSL object.
What you've stumbled on here is kind of incredible though, especially the fact that it's not consistent and contradicts their own docs about what happens when you call .asText.get() (https://docs.gradle.org/current/javadoc/org/gradle/process/ExecOutput.StandardStreamContent.html#getAsText--).
I wonder if it's due to the caching behaviour that they describe here: https://docs.gradle.org/current/javadoc/org/gradle/api/provider/ProviderFactory.html#exec-org.gradle.api.Action-
@dgregory @jason The other thing I really, really wonder is whether it happens at all on non-Windows systems.
The really disturbing thing about this to me is I wasn't doing anything "weird"! I wasn't pushing disparate features together or violating abstractions. I just wanted to include the git hash in the build, or if git is not present, the string "unknown". That's something many projects do, I did it in the most obvious way, and there's no other well documented way.
@mcc @jason
Yeah for sure, I'm not aware that there's a more idiomatic way to do it or anything like that.
I wasn't able to replicate this on an M1 Mac w/ Gradle 7.6, i.e. I consistently get "unknown" as the result of "printGitSha". I have no idea what version of Gradle is used in Android Studio though!
@dgregory @jason Did this happen in the Tusky build.gradle or did you create a build.gradle to test. Because in my testing the problem does not reproduce in simple examples. I made one myself (linked in bug) and it does not hit the problem even on Android. I am worried it might be a race condition, which is about the most cursed possible thing to be possible in a build system.
@mcc yup.
On the plus side jetbrains are pretty responsive to bug reports and genuinely improve their products with each release.
On the negative side, your problem is with gradle.
@mcc it was so much worse when everything was based on Eclipse, if you can believe it
@SpindleyQ I have used Eclipse and I can believe it.
@mcc I've long wondered why I kept bouncing off of Android development but after following your trials and tribulations I'm surprised I ever got as far as I did.
@mcc Wow. That is indeed cursed.
@mcc If you stare into the void long enough it will sometimes throw an exception back.
@mcc This sure sounds like gradle. I hate it but you should see what it was like when we had to use maven.
Java tooling is unbelievably cursed. Not really surprising when you consider that Java is a fundamentally idiotic language.
@mcc Gradle alone is enough to put me off of Android development. Is Bazel a good option by now?
@BoredomFestival I literally didn't know bazel was an option on Android until this moment.
Is this true? Gradle is way too slow already
@mcc I thought it was, maybe I'm wrong :-\
@BoredomFestival sorry, what I mean is I was looking around on Google and I found lots of indications you *can* build Android apps with bazel, but I could not tell how well supported it is.
But then I found this blog post by Gradle in which they claim Gradle is much faster. Gradle is of course a biased source so I was wondering if that was true.
@mcc@mastodon.social @BoredomFestival@sfba.social Yeah there's definitely tooling for it internal to G but there's always a huge delta of what works with bazel at G and what the community plugins can do
@LunaRogue @BoredomFestival what i have heard is that the Original Sin of all google developer tools is that everyone inside google is using something called "Blaze" for everything, and for some reason that can never be released open source, so the entire Google external ecosystem is trying to use a patchwork of things to simulate Blaze, of which Bazel, some kind of open source reimplementation of Blaze(?), is the most recent, least unusual, and least complete(?) attempt
@mcc@mastodon.social @BoredomFestival@sfba.social Yeah, that's how it goes there. They've got some internal thing, and it's good for their use case but too tied to their proprietary stuff, so they open source / reimplement for FOSS some of it, but eventually they hit a point where their internal thing is better for half of cases and the FOSS thing is better for the other half and everyone loses.
@mcc @LunaRogue Bazel is (basically) Google's open-source version of Blaze, and it's really quite well thought out for the most part... with a few huge problems that make it unsuitable for a lot of work outside Google, because of design decisions that make since inside Google's monorepo but not anywhere else. (e.g., the team's steadfast refusal to allow it to build static libraries as a final build product, which has been an open bug since 2016)
@mcc @LunaRogue From an overall design point of view, Blaze/Bazel is (by far!) the best build system I've used in my 30+ years in the industry, but Google has no incentive to finish all the rough edges that exist for arbitrary projects, so it seems likely to stay at the stage of "good enough for Google's projects but not much else", unfortunately.
@mcc @LunaRogue (I would looooove to see LLVM find a way to ditch CMake and move to Bazel, but at this point I'm pretty sure that's tantamount to "I want a pony")
@mcc Even if Gradle built everything instantly, it's just too weird and hard to grok to be worthwhile (see: "Groovy" as the scripting language).
Also note that I work on a project that uses CMake as its primary build tooling, so I know a thing or two about terrible build systems.