The Breakroom

In the last post, I talked about overcoming some issues with Wallaroo‘s main gallery view, but unexpectedly even the simple wallpaper detail screen held its share of challenges!

The biggest surprise for me was how much trouble I had implementing the paging view to swipe left and right between wallpaper variants. In UIKit it’s trivial to enable paged mode on a UIScrollView to get this behavior, but I couldn’t find a way to do it in SwiftUI for some reason. I ran across a bunch of tutorials that were manually implementing panning gestures and paging and I was flabbergasted. Did I really need to do all of this myself from scratch?

I tried a few different open source paging view implementations I found on GitHub but they were all lacking in one way or another.

The most glaring problem with them tended to be that the scrolling physics felt entirely wrong. The swipes didn’t have the same momentum that Apple uses everywhere else and the resistance on the rubber band effect (if it even had it) was never quite right. I started going down the road of adapting one of the better ones with more correct physics after I discovered this gem of a project which meticulously deconstructed the formulas necessary to get an Apple-like scroll view feeling. Still, though, nothing about this approach felt right.

Eventually I decided that maybe I should just wrap a UIScrollView to accomplish what we needed and started to dive into that.

I was pretty far into this when I accidentally stumbled across a Stack Overflow comment noting that SwiftUI actually does have a native paging view – it’s just cleverly hidden as a TabView style!

OMFG.

I wasted most of a week on approaches to this before I managed to discover that, lo and behold, SwiftUI actually does have a way to do it – and it’s a single line of code: .tabViewStyle(.page).

I was supremely pissed off at myself for somehow missing this. Despite all of my searching, I never managed to notice any reference to it in the documentation and apparently didn’t remember anything about it from any of the WWDC videos I had watched, either. I suppose perhaps my search terms were too laced with an assumption that it’d be a part of ScrollView somehow or that it would exist as its own view?

I soon learned that SwiftUI didn’t always include support for a paging view – which is why there were so many outdated tutorials and open source projects out there for me to accidentally find instead of the official API. Argh.

SwiftUI changes so much every year and I can’t count the number of times I found an accepted Stack Overflow answer for a thing that was completely wrong as a result. (This is also true of Swift itself – although it’s getting better over time as they’ve become more resistant to source breaking changes.)

There’s also something off about how SwiftUI documentation is written and organized. It frequently feels next to impossible to find the name of whatever view modifier you might be needing unless you more or less already know what you’re looking for. To make matters worse, the fact that SwiftUI’s view modifiers almost all exist as function extensions on View means just pressing the period key and browsing autocompletion suggestions tells you nothing about what might or might not make sense to use in your current context.

The plethora of tutorials, Medium articles, YouTube videos, and books also mostly have a feeling of “magical thinking” to them – as if you just need to memorize incantations to spells rather than learn any of the actual science behind the effects you are trying to achieve. I find that aspect of the SwiftUI ecosystem extremely frustrating and disappointing.

All together these things make SwiftUI feel very difficult to learn and master.

So anyway, after discovering TabView‘s support for paging, I threw away everything I had been working on for a week and used it instead.

Unfortunately it is extremely buggy.

I don’t want to get in to every single problem I ran into with it or this blog series would be about three times longer than it is already, but suffice to say there were a lot. My source file has 6 different bullet points of issues I ran into!

Luckily I was able to workaround them all (one dumb issue was needing to wrap the TabView inside of a ScrollView with scrolling disabled in order to get it to stop jumping around when it first appeared which then of course ended up requiring a second workaround to fix an issue with that workaround…), but the worst, by far, was a memory leak/retain cycle!

We discovered the memory problem around the time I was working on the AsyncImage replacement I mentioned earlier. I had implemented various caching schemes for the images and I kept modifying my approach because it turned out the app was eating RAM and the most obvious culprit was the cache. In fact the initial reason I ripped most of the caching out and revisited it was in an attempt to hunt down this leak!

Unfortunately even with all caching removed I was still seeing a huge memory leak somewhere. I tried using Instruments to find it, but I had no luck deciphering where it was coming from – other than to see confirmation that memory was indeed leaking like crazy.

I spent two days on this problem.

I don’t mean two work days – I mean pretty much two entire days stopping only for food and sleep.

I was trying everything I could think of to narrow it down. I first started by commenting out random stuff and manually running through the very long sequence of events I had found that could reliably consume the memory. Over… and over…. and over… (And over and over and over…)

Eventually I discovered (around 1am on the second day) that using a selection binding on the TabView was keeping the entire TabView instance alive somehow even when the view that contained it was destroyed.

So over time, after viewing a bunch of wallpapers and scrolling through the variants, all of those huge images were staying in memory even when you went back to the gallery view and loaded a different wallpaper! Slowly, with regular usage, the consumed memory climbed and climbed until iOS killed the app.

My paging view was its own standalone view that had a TabView inside of it. To expose the currently selected tab to the wallpaper details view, I passed in a binding which was in turn used by the TabView. Somehow, though, this is what was causing the problem because when I commented that part out, it wouldn’t leak – but I needed the selection binding to be passed along from the detail view to the inner TabView somehow!

On a whim I tried doing something more indirect and added a new state variable to the paging view which would hold the selection and passed that to the TabView as a binding instead. Then I used .onChanged to note when the selection state variable changed and updated the original binding that was passed into my wrapper view from outside. This appeared to break whatever retain cycle was happening.

I don’t know exactly why this worked (I can only speculate), but some part of my tired and sleepy brain thought to try it and it turned out to solve the issue in the end – but not without some pretty serious frustration and anger.

I feel like I should send Apple a bill for the pain and suffering I endured on this one.

Since this caused me so many problems, I decided that I would share my PagingView code to maybe save someone else some time or see if anyone has any better ideas. (And if we’re really lucky, maybe a SwiftUI engineer will see this and fix it!)

Now that this rather unexpected battle with the paging view was behind us, we were surely on the homestretch! Wallaroo was nearly fully functional – except for one minor detail…

Now that Wallaroo had a unique and good-looking main gallery view along with nice parallax and blurs and everything, my focus shifted to optimizing scrolling performance.

The parallax and lazy custom layout seemed to perform just fine, but rendering the glyphs/buttons on each tile view as well as the actual wallpaper image seemed to be slowing things down a lot.

When I commented each of these bits of code out in turn, scrolling would become silky smooth, but enabling just the glyph/button overlays or just the wallpaper images would result in unacceptable stuttering on my 2016 iPad Pro. Hmm!

Eagle-eyed readers might have noticed earlier that I said I was using AsyncImage “at the time.” Eventually I replaced it with a custom implementation on my quest to achieve smoother scrolling and then extended it a bit more to solve another problem we ran into.

It seemed (at least on the earlier iOS 16 betas), that AsyncImage did not prepare the image for display before rendering it. That meant that when two or three images all loaded during the same view update cycle it would cause animation hitches.

I solved this in my custom view by using a .task modifier so I could async load the image data myself, then convert it to a UIImage, and then call the async version of UIImage.byPreparingForDisplay all before finally assigning the SwiftUI Image instance that would ultimately be displayed to a state variable.

Around this time Craig also realized that as I worked throughout the day my computer was hitting the image server a whole lot and this probably was not going to scale very well once it went live in the App Store.

To address this, I adjusted the URLRequest for the images to always return from the URL cache directly when possible, but ultimately we fixed our server to set the correct HTTP caching headers so the URL loading system stopped rechecking for freshness all the time and trusted that the cached data was good.

If Craig hadn’t been watching our server logs at the time, I’m betting we’d have totally missed this and then had to deal with a flood of server traffic on launch day!

(Aside: It may sound like we arrived at all of this in a matter or hours, but of course we didn’t. I spent a ridiculous amount of time off and on over weeks trying various caching schemes for my custom AsyncImage that prepared the image in different ways, kept the prepared image around keyed by URL, used NSCache or a simple Dictionary, used a Swift actor, etc. Happily I was able to get rid of all of that complication once the required pieces finally fell into place. This sort of thing happens a lot and I often spend a bunch of time on things that turn out to be dead ends. I tend not to stop the moment I get something working, but instead keep thinking about it for a while trying to simplify or get rid of as much code as I can. Sometimes the results are good as in this case, but other times it can go badly where stuff becomes much too specific to a single situation and ends up causing problems for myself later. Such is the nature of evolution, I guess?)

Another of the limitations of AsyncImage that I was able to work around in our custom implementation is that AsyncImage only delivers the final Image after the load finishes and not the original data.

We wanted to be able to access the raw downloaded data of the big images as seen in the full wallpaper detail view so we’d have it when saving or setting the wallpaper. I built a way into my custom image view to get the downloaded image’s raw Data as well as the prepared Image to avoid needing to re-encode the image or download it again separately. There are other ways to solve this problem (I tried a few), but it seemed the most straightforward to do it this way since it ensured the image data was only ever downloaded once.

After getting the thumbnails to load without a hitch, I moved on to the troublesome issue with the glyphs.

There are at most three symbols on each wallpaper gallery tile: The favorite heart, the multi-variant indicator, and the lock (which only shows up on some of them when unsubscribed). When these overlays were enabled, it caused a ridiculous amount of stuttering.

I had a suspicion as to the root cause: Gedeon had wanted the glyphs to be rendered with a custom shadow using a blend mode that didn’t interfere with the semi-transparent favorite heart but also worked against a wide range of wallpapers. This effect was first implemented early in the prototyping stage and hadn’t been revisited.

Sure enough when I disabled these effects the scrolling was perfectly smooth. I tried a whole bunch of things to get this to perform better until finally stumbling on the fact that I was using a .mask modifier on the glyph views in order to get the appearance we wanted. When I came up with a much smarter way to make the effect work without masking, the performance skyrocketed. I guess the lesson here is: Don’t use view masking if you can help it!

This highlights one of the weakness of SwiftUI. It is very hard to debug this sort of thing or to get a sense of where time is being spent by the engine because so much of it is inaccessible and outside of our control. Hopefully these things can be improved somehow in the future. For all I know, maybe there are already ways to use instruments to diagnose problems like this – although personally I find a lot of the instruments to be inscrutable and it ends up taking me just as long to trace through a problem with them as it does to hunt around with educated trial and error. But maybe that’s just me.

With most of the issues surrounding the main gallery view taken care of, it was time to finally move on to the much more straightforward wallpaper detail view! How hard can it be?

At the end of the previous post of this series, Gedeon had just suggested that we add a parallax effect while scrolling through Wallaroo‘s gallery view.

There’s nothing inherently complicated about parallax – it’s just moving a background layer at a different rate from the foreground. The real question was how the heck I was going to do add it without ruining the simplicity of the code I had built so far.

After I stopped despairing, I realized I already had the information I needed to do this: I knew where the frame of the overall visible view was on screen, and I knew the frame of the child tile view. That’s all it takes to come up with some kind of an offset for a layer. The problem was, that information wasn’t really where I wanted it to be – namely somewhere inside of the ReleaseTileView.

I started by trying to compute an offset in the main content view and passing it into the ReleaseTileView, but that didn’t feel like a very “SwiftUI” way to do it (although it worked and was relatively straightforward). Instead, I spent some time trying to research how problems like this are meant to be solved in a way that doesn’t fight (or at least publicly embarrass) the framework.

Some of the scenarios I thought about where things like: What if I wanted to have this parallax effect in some small cutout deep inside of the tile view and not on the whole background? How would the information get all the way there? Would it be passed from view to view in their initializers? What if Gedeon suddenly decided that only some of the views should do the parallax effect while others should not? What if that condition was something specific to an individual release? Etc. While some of these weren’t likely scenarios, I wanted to make sure I did it “right” at least insomuch as I could determine what that meant.

What I ended up going with was pretty simple: I decided that knowing where the containing gallery’s frame was would likely make sense as an environment value, and so I added the visibleGalleryFrame that was computed in the code snippet in part 1 as an environment value for all of the child tile views to use. To get the parallax to work, the tile views measure the frame of whatever area they want to apply the parallax effect to and compare it with the environment’s gallery frame and offset things accordingly.

While I’m sure there are many good ways to do it, it seemed to me that this approach is in line with how SwiftUI wants us to think which is to consider which pieces of information are part of something like the “world” that a view lives within vs. the information a view is meant to be displaying.

Of course, even this is still a bit fuzzy. I think it could be argued, for example, that putting the wallpaper object itself in the environment rather than passing it to the initializer for each child view could also make sense! I’m not sure where to draw the line with this sort of thinking, to be honest. There’s really nothing stopping almost every value normally passed as a parameter from being passed in the environment instead, but there’s something about that which feels like it’s terribly wrong – I just can’t articulate exactly what the rules should be.

It feels like the only thing preventing people from using the environment more often (and perhaps “incorrectly”) is the fact that defining a new environment value involves a bunch of boilerplate in an EnvironmentValues extension! It doesn’t seem ideal to only depend on that friction to guide behavior, though, especially since I’d love to see some kind of new Swift syntax or feature that could eventually make the boilerplate go away!

While the sweet parallax effect was a nice touch, there was still an annoying flaw with the gallery view that jumped out every time we scrolled it: The thumbnails sometimes took awhile to arrive over the network leaving an empty hole while loading.

From my time playing around with Mastodon, I remembered that they had a solution to this which displayed a blurred image while the full thumbnails loaded. I didn’t know how that worked, so I dug into their code a bit to find out how they did it. I discovered that some very smart people came up with a way to encode a blur into a relatively small string using math I do not even remotely understand myself.

I shoehorned the open source BlurHash code into a SwiftUI view that could take a hash string and a size and generate an Image from it. Once that was working, Craig modified our backend to compute real hashes for all of the thumbnails and images and included them with the catalog of JSON data that the app downloads when fetching new content. The whole process was surprisingly straightforward and, in our opinions anyway, is much nicer than having a static bundled placeholder image!

Adding all of these bells and whistles makes the app look pretty, but does it perform well? Wait until part 3 to find out – or just download the app and check it out yourself!