While there are many ways to work with different AI tools, one that originates from design can be particularly effective: AI as interaction materials.
What do we mean by materials?
While the knowledge of the materials used is prominent in all kinds of designs in the physical space, this is a topic more in the background for the digital space. At the same time, Google’s Material Design has made any search on this difficult. Yet, I’ve always believed that with the right framing it’s an extremely useful concept.
An interaction material is an abstraction of the technical components of a digital interaction where we focus on the cognitive properties (how we perceive it), technical capabilities (what it can do), and technical limitations (what it can not do).
This isn’t a general article so I won’t get into too much detail, but for example these can all be considered interaction materials:
- Web browser rendering engines (Firefox, Chromium, WebKit, …)
- Libraries to perform searches (SQL, Elasticsearch, Lucene, …)
- Programming languages (JavaScript, PHP, Java, …)
- Animation libraries (GSAP, Lottie, Three.js, …)
- And of course: LLMs.
Why interaction materials matter
The simplest way to explain interaction materials is to think of a brick. To effectively use a brick to build things we need to know its properties and its limitations (i.e. max load, temperature limits, etc). Knowing different kinds of bricks allows us to pick the right one for the thing we are building right now. We don’t need to know how a brick is made to know its properties and use it effectively in designing a building.
In the same way good architects don’t need expertise in building craft, product leaders don’t need to know every technical detail about how their materials are made to leverage them effectively. A good stopping point is when we reach the ability to know something as an interaction material: cognitive properties, technical capacities, and technical limitations.
We don’t need to know how to build LLMs or other AI tools to be able to make good decisions about using them. We do need to know them as an interaction material. Anything shorter than that, isn’t enough.
How to learn about AI as a material
It might not be intuitive how to get a grasp of something at an interaction material level given it’s something intangible. But there are many techniques to gain the knowledge needed:
- Try it out as many variations as possible
- Try it out… on something real(ish)
- Read resources at one level deeper
- Check the libraries
- Talk with people with direct experience
Try it out as many variations as possible
Especially now that we are in what can be framed as the pioneering years of GenAI, it’s important to try out as many tools as possible to find exactly how they work and how they are solving problems (and possibly find some that fit your needs). Check their free plans, check the open-source options, see directly how to go from zero to using them.
To explore their cognitive properties, try to use them a bit. Follow their tutorials and documentation to start with, and ideally some writing by people that have used them before. See what choices they made in how to interact with these models and tools. Is it text only? Guided? Other modes? How fast? How does it feel using them? Try to get a sense of the choices they made to reach this specific interface.
To explore capacities and limitations try to do things that are very specific, or very generic. See what the tool does. Try things you wouldn’t do if you had a goal. The goal right at this phase is stress-testing to see if it breaks, and if it does, in which ways.
Try it out… on something real(ish)
This might seem obvious, but the next step is to try the model or capability on something real in the product’s context, but not directly on your product. This means thinking: “How would I use that for something actually productive?”. You can reach out and see how other people are using it or look for examples in the industry. Try to find people that explain exactly how they use it in your context. Focus on real examples with documentation, not people that just tell what would be ideal or stay too abstract.
While the goal is similar to the previous one, trying to contextualize in your own work is important to get a grasp of the nuances. Often I try things in a generic way and think “oh yeah, it does X” but then when I actually try to apply “X” to something real… that’s where I find the limits of an LLM or Gen AI product.
Even if you don’t plan to actually use the LLM or AI product long term, it’s a good thought experiment to find ways to think about horizons and strategic improvements. If you are working on a product, you can also think about personas and customers of that product and imagine how they would use AI or things outside your product to solve their problems.
Read resources one level deeper
While going all the way on how to “make” AI tools is not feasible for everyone, often it’s good to go one level deeper. Move from the “user” to the “last mile builder”. How are they building it in their own products? Can you find articles of people that are implementing libraries that explain how to do it? Can you read forums where they are discussing the challenges they are finding?
The key here is not to learn how to make the AI tool (again: if that’s your thing, please do!) but to see the rough edges just under the surface. To understand how the thinking works. Something that might look easy and simple for the final user might be a lot of swearing on developer forums and months of work to fix it instead of just a plug-in library.
Check the libraries
This is something you’ll likely find out by doing the previous step: what are the libraries people are using? Look into them, and the specific beginner tutorials in their documentation on how to get started.
Talk with people with direct experience
This can happen earlier, but it’s also important to talk to people that are using these tools, not just people that write about these tools. Is there any team inside your company that is experimenting already? How are designers that have done it before thinking about it? Engineers? Product Managers? Yes discuss also with more senior people that define strategies of course, but don’t miss the people that have direct experience.
There are of course many other ways to explore that can lead to learning a new interaction material like AI, what’s important is that you focus on its three key factors: cognitive properties, technical capabilities, and technical limitations.
It could also be useful to prepare a short learning plan before starting. The above might get very meandering given how open the discovery process can be, so having an idea of the specific actions you want to take (i.e. “Try 4 different LLM customer-care SaaS companies”, “Identify 2 libraries I could discuss with my engineering team”) and maybe time-box them can be very effective.
I also assume a lot of the people reading this article might already be half-way through some similar learning path. In that case my advice is to use the three key factors and the learning steps above and see if there are any gaps you feel might be useful to cover.
The important thing is to reach a rounded understanding of it: the knowledge of an interaction material.
Thanks to Saielle DaSilva for reviewing this article.