4TY logo

ORT-3 - Open Rotation Test in 3 Dimensions

History of (the variants of) this test

Of the many tests of spatial ability, one of the more common and well-known is "the Purdue test", or more fully, the "Purdue Spatial Visualization Test: Visualization of Rotations". It goes by many other names, and some acronyms. There is a Wikipedia article about it, which at time of this writing says:

The Purdue Spatial Visualization Test-Visualization of Rotations (PSVT:R) is a test of spatial visualization ability published by Roland B. Guay in 1977. Many modifications of the test exist.

The test consists of thirty questions of increasing difficulty, the standard time limit is 20 minutes.

Those statements are - or may be - accurate as far as they go. Personally when I went looking for the test I found it pretty easy to locate three versions. Two of them (Guay's original, and a slightly tweaked version described below) did consist of 30 questions but a third contains just 20. This is "The Purdue Visualization of Rotations Test" as the authors - Bodner and Guay (1997) - termed it; in their paper they also called it simply "the ROT" or sometimes "the 20-item ROT test".

At the time Bodner and Guay wrote their 1997 paper - a couple of decades after Guay's 1976 paper titled "Purdue Spatial Visualization Test"* in which he published/offered his original test - it had become clear that the original 30-item test had some problems. In particular, some of the test items, that, it was originally thought, should be relatively easier, were instead showing up as very difficult. The problem is easy to discern at a glance and is also discernible upon close reading of later literature (more below). In re-releasing their new test, Bodner and Guay removed 10 items including a large preponderance of the more-or-less "most offending" test items. Leaving the shorter test, the 20-item ROT described in their 1997 paper. They do not explain their choices in their paper but given that many of the removed 10 items were "problematic" it sure looks like one of their intents, was to ameliorate.

Bodner and Guay freely offered their 20-item ROT test for general use. Quoting from their 1997 paper:

"Copies of the 20-item Purdue Visualization of Rotations test and a scoring key can be obtained from the first author. They can also be downloaded from this journal's abstract page for this article. Permission to use this test is freely granted. Reasons to use this test are many and varied. It can be used to diagnose ... It can be used as a research instrument ... It can be used to probe changes in gender effects on spatial ability. It can be used as the basis for evaluating courses ... It can be used to probe students' perception ..." [et cetera]

That's a pretty generous, expansive permission and set of sample/suggested uses. It could be tempting to just use it.

However, Guay is no longer on the scene, and many of the various test materials Guay produced, including ones identical to the freely-offered 20-item test just described, tend to include a copyright notice naming the Purdue Research Foundation as copyright holder. I have found the foundation folks good to work with, but my interaction with them has been minimal. I don't know what would happen if a person or organization tried taking up Guay on his offer. A cautious consideration of possible outcomes with an eye to likely characteristics of entities like profit-generating commercial arms of universities, leads to wariness of a threat of a litigious entity claiming copyright.

*I don't know why Wikipedia chose 1977 rather than 1976 as the date of publication.

Isometric Ambiguity

Above and beyond copyright concerns. Even after Bodner and Guay deleted the "most offending" items from their test, and freely offered it, a problem with it remained, and still does, in all the successor/variant versions I have encountered. The problem is rooted in the historical development of engineering drawings and technology. All three of the aforementioned tests use a drawing technique called "isometric drawing", fairly common at one time (in the previous century) and in some professions - though it never spread to others, due to its limitations. Isometric drawing - actually had (and still has) some advantages, but it suffers from a big disadvantage called "Isometric Ambiguity"; a problem that is fairly well-known and quite well-documented.

Prior work/attempts to ameliorate the deficiencies

Yue (2008) redrew the original drawings from the older (1976) version of the PSVT-R and in so doing discovered a number of outright errors in the original isometric drawings. But even after fixing those errors, many of the isometric drawings were just too obscure to understand - that is, they suffered badly either from isometric ambiguity, or from a relative of it: the simple inability to even begin to understand "what some of those drawings even represent." Certain isometric drawings, are just plain hard to figure out. This is another known problem; very well-known in fact; mechanical drawing classes used to focus heavily on bringing students up to speed, training them to be able to "see what the drawings mean."

Among other things Yue played around with techniques such as applying a wood-grain pattern to the drawings. It didn't work very well - also it violates one of the fundamental precepts of this sort of mechanical drawing: you should be able to represent the object, using just black-on-white line drawings. No fancy shading - nor shading of any kind - allowed/accepted, if only for the reason that photocopying drawings (and/or duplicating them using more current means, like scanners and cell phones), create bad results/artifacts (and at bad times); for example streakiness, or certain dark gray colors getting turned into all-black. Basically spoiling the copies for use.

Yoon* in her 2011 dissertation titled "Psychometric Properties of the Revised Purdue Spatial Visualization Tests: Visualization Of Rotations (The Revised PSVT:R)", did what the title of her paper suggests: analyzed the psychometric properties of the 30-item test. I don't know why she chose the 30-item test rather than Guay's 20-item test, which seems to fix some deficiencies. I guess it was because the 30-item test had over the years become a standard.

In any case, Yoon, using straightforward testing techniques, and using redrawn figures (without the errors identified by Yue, but still using isometric drawing) identified the difficulty of each test item, and reordered the (slightly) updated test in order of difficulty. Quoting her:

However, the results based on the CTT revealed that the difficulty level of the items was not consistent with the order of items arranged by the degree of complexity in rotation. For instance, Item 24 (P = .729), which requires two different rotations, was easier than Item 6 (P = .665), which requires a single 90° rotation. This implies that the level of difficulty perceived by respondents might not simply depend on the complexity of rotations.

A glance through the test shows why many of the test items that Guay had figured, would be relatively easy, were in fact found to be difficult, frustrating test-takers. The problem is simply that isometric drawing really isn't a "natural", human-oriented representation. Meaning that in the real world, things "just don't look like that."

There is another problem with Yoon's work and the resultant test. Yoon makes extensive use of Item Response Theory (IRT), using it for example to analyze the psychometrics of the test items - the central point of her dissertation - and then reorder them in order of difficulty as she determined it using statistical techniques. However, she herself points out in her dissertation that

In contrast to CTT, IRT is based on strong assumptions: (a) all items on a test measure a single construct (unidimensionality), (b) responses on an item are not dependent on the other items but dependent on respondents' ability level (conditional independence or local independence)...

Unfortunately both these "strong assumptions" are violated in this case: meaning in the three cases of 1) Guay's original, 2) Guay's revised, and 3) Yoon's revised versions. Clearly, both by carefully reading Yoon's own work, and also just by looking at the tests, you can see that there are two constructs in play, not just a single one. Here are the two:

  1. What the test intends to test - the test-taker's ability to visualize rotations of objects.
  2. A confounding** construct: the test-taker's ability to disambiguate the sometimes-infernally-aggravating drawings. Before you can even begin to address the above (figuring out how an object might look after some rotations), you have first to figure out what the heck the object even is - what the isometric drawing is representing.

That second skill isn't always easy and it can and does take a lot of time, fairly often slowing progression through the test to a crawl. Fairly often, test-takers, baffled by what appears to be incomprehensible, just give up.

The ability to disambiguate confusing (even baffling) isometric drawings, could once - back in the day/millennium when such drawings were in more common use - have been important, valuable. But just as clearly it is a separate skill from being able to visualize rotations. A simple thought experiment supports this claim: imagine presenting an adapted version of the Guay test to blind test-takers; giving them physical models, 3-D objects, posing to them the same questions: "Which one of the following 5 will result after you've rotated the item in the demonstrated manner? These test-takers would have zero ability to disambiguate 2-D isometric drawings as they're blind. They would not suffer from the same disadvantage as a sighted test-taker who needs to spend, say, 90 seconds figuring out "what the heck's this supposed to mean?"

Of course a thought experiment is just a thought experiment - it really doesn't prove anything, so to be of more than speculative/inceptional use, it should be followed up by real experiments. In this case a variant of this test is already in fairly widespread use - adapting the test using physical models which the test-taker handles inside a (non-see-through) cloth bag, rendering them "blind" as to the contents. This could be an avenue for further research, if it hasn't been done already (I have not checked).

The other assumption - Yoon's "strong assumption" (b) quoted above, is also violated in this case. The violation is a little less clear as it involves a (fairly well-known) fact of spatial testing - the (often very) rapid learning we see as people take tests. Specifically in this case, after seeing and attempting to disambiguate various thorny/baffling isometric drawings, you get better at doing it. So in a real testing environment, some items do depend on other items, in the sense that the item you are looking at right now is easier than it would have been just an hour ago. The "dependency" is that your exposure to prior test items does affect - in a big/significant way - your performance on the current question, the one you now face.

*Full name: So Yoon Yoon - she goes by "Yoona"

**I use the word "confounding" here in its lay sense; in the field of testing and statistical analysis that same word has a special technical definition; perhaps I should use a different term, but this is clear to most people including non-experts.

A digression - spatial ability testing in general

Spatial ability - which the ORT-3 is designed to test - is in certain circles, a "hot topic" and contentious. In particular the everyday fact that "men do better" at mechanical tasks and fixing things around the house, or in academic/research terms, males consistently and regularly perform markedly better on tests of spatial ability (not to mention do better in college courses, disciplines, and careers involving spatial ability), causes a lot of consternation. A common response is to posit - or claim, assert, or insist - that spatial skills are trainable and "highly malleable".

A study this is perhaps most commonly cited in support of this, is Uttal et alia's 2013 meta-analysis of 217 spatial testing studies. The "takeaway" conclusion, that appears to get most of the attention and be most commonly quoted, is this:

[S]patial skills are highly malleable, and ... spatial training is effective, durable, and transferable.

But the Uttal paper itself mentions that many of the studies they reviewed, appeared to demonstrate just the opposite.

In fact the Uttal paper is long, contains a lot of information, provides some good insights and ideas, and makes a number of conclusions; some of which are relevant to my own effort, and to a practical, meaningful test of spatial ability. Here are some quotes from the Uttal paper:

Our analysis suggests that reaction time improves more than accuracy does.

The considerable improvement in response time in both the treatment and control groups indicates that people get faster at performing mental rotation tasks with practice.  The improvement in response time as a function of minimal practice appears to be so strong that for studies that used response time as the only measure, treatment groups did not improve significantly more than the control groups, p > .73, even though treatment groups did improve a great deal.

Why did our meta-analysis reveal that transfer is possible when other researchers have argued that transfer is not possible[?] One possibility is that studies that planned to test for transfer were designed in such a way to maximize the likelihood of achieving transfer. The attainment of transfer effects often requires intensive training[.] For example, many studies that achieved transfer effects administered large numbers of trials during training ... or trained participants to asymptote[.] Transfer effects also must be large enough to surpass the test-retest effects observed in control groups[.] Thus, although it is true that many studies do not find transfer, our results clearly show that transfer is possible if sufficient training or experience is provided.

Why are spatial skills so malleable? Our results raise the important question of why spatial skills respond so well to training. One important factor may be the relative lack of experiential opportunities in everyday life to practice spatial skills.

In some cases, the average magnitude of improvement in the control groups was greater than that observed in the intervention groups in studies of other kinds of educational improvement programs.

Improvement in the control groups, therefore, is seen as a measure of the effects of repeated practice in taking the assessments, familiarity with the mode of testing[,] ... and specific practice that could be gained from the assessment items themselves, all independent of the training intervention.

[P]articipants in the control groups may also learn something that is more general than simply becoming familiar with a specific test. The spatial skills of participants in the control groups may have improved because taking spatial tests, especially multiple tests, can be in itself a form of training.

[T]he sheer magnitude of the improvement in the control groups suggests that something more than retesting effects took place.

[S]erving in a control group promoted an increase in spatial skills. Taking spatial tests can be interpreted as a form of incidental training, although it is not as effective as training that is designed to improve a specific spatial skill. Our explanation for the learning that occurs in the control group is similar to our explanation for the large magnitude of training effects in general: because participants are relatively unfamiliar with the type of reasoning that spatial tasks demand, they can improve a substantial amount with relatively little training. Even the experience of serving in the control group can be sufficient to prompt meaningful spatial learning.

Of course I have quoted selectively, my interest is "motivated" by my desire for a free, high-quality, fun/attractive test of spatial ability. There's a lot I left out of the above - for example one small part about a related field: navigation, saying in passing that "humans possess sophisticated navigation skills". If you want a more complete - and potentially less biased - understanding of the Uttal paper, read it - it's interesting.

As I read the Uttal paper, especially the above quotes drawn from it, while bearing in mind Guay's work, it's plain to me that part of the rapid improvement of test results, is due to test-takers improving their ability to decode the isometric drawings used in the tests. Of course your viewpoint may differ but I've read enough and played around with the materials enough, to become convinced that a "cleaner" test is in order.

Practice effects, rapid and not-so-rapid learning

If it's true, as I claim, that the isometric drawings used in the Guay variants are "problematic", why has this not been known, recognized, and/or ameliorated already?

Well, first of all, perhaps it has. I am not any kind of expert in spatial intelligence research, I don't know the field. Perhaps someone has "been all over this" already; I wouldn't know. (Nor do I have time nor desire to find out - my own interest is rather more pragmatic).

But there is something else that goes on in the real world, that suggests another explanatory possibility, a plausible reason. Just as the study above and others, also personal experience with these tests, suggests: people - generally unused to isometric drawing and in most cases with zero prior exposure to it - are at first flummoxed by it, but then - at least in many cases - are able to figure it out. Folks become used to it, get better at overcoming its limitations and confusing aspects described above, and begin to see it as commonplace.

Here is an example of people's ability to overcome limitations of techniques and instruments, from another field. This is a photograph of both a crater and a dome, on the moon:

crater and dome
Image from Lunar Reconnaissance Orbiter Camera, see https://lroc.im-ldi.com/images/378

This example comes from the field of planetary geology. Note that what looks - to the normal/untrained eye/observer - like a crater, is actually a dome (yes, it really, truly is in fact). Also what looks like a dome, is really a crater. Planetary geologists who teach and/or deal with the public are well aware of this, and (typically) themselves went through a phase early in their studies/careers in which they had to develop a knack for seeing the craters as craters. I am confidently told by such experts/instructors that the problem of seeing craters as domes and vice versa (like I and most other people do) is no big deal, it's not a problem. It shouldn't stand in the way of anyone's career ambitions, nor is there any program of study, nor remediation, required. After a while you just "get it" - the skill or ability becomes second nature.

I'm still terrible at this moon/dome/crater stuff, but as I work more and more with isometric drawings (and getting back to the topic of spatial ability testing) I can see how this happens. When as a neophyte I looked at some of those test questions showing "infernal", apparently bizarre isometric drawings, they were highly confusing. But now that I am more used to them they are not so.

It seems plausible - even likely - to me that what goes on in the field is, the persons who maintain these testing instruments, are themselves "at ease" with isometric drawing. After their years or decades of experience, to them it's no big deal.

My own background is different - I'm no expert in matters spatial. I deal with the eminent examples of neophytes (children, tweens, teens) and my goal for any test that I will use, is that it will accurately measure - to the best, "cleanest" extent possible - a person's actual spatial ability, whatever that may be and however one may define it.

Definitions of spatial ability

People recognize the importance of spatial ability but struggle to define it. Not for lack of trying! There's voluminous literature on the topic. The best - and possibly most cited/influential? - breakdown of spatial ability I've run across comes from Thurstone, who suggested/defined three spatial factors:

Personally I understand those first two - S1 and S2 - fairly well, but the third gets a little more difficult. The language is harder for me to understand. This is common to the literature - simply, it's hard to put or describe, in words, that which is fundamentally a non-word-based domain. Or to say it another way, spatial problems, entities, relations, and abilities - they fall into a category of their own, it's just not easy - or maybe even it's impossible - to put it into words.

Newcombe and Shipley (2015) sketch out a 4-way breakdown of spatial skills:

and go further, describing spatial thinking and relations (in part) like this:

I think there has been a lot of work done in the field besides these two, some of which I have omitted here out of simple mercy for the reader. My point in including this material is this: nowhere in the above lists - nor in others that I have seen - do I find listed as an important or desired skill "ability to decode - meaning overcome the established deficiencies of - particular graphical representation techniques." On the face of it, this ability, which I have found from my own experience to be real and significant, is something you'd rather not have to have at all. Instead, just use a better, different representation technique. Fortunately that's easy.

Outcome, the design of this test

It is actually possible to redraw any drawing originally drawn using an isometric representation, using more modern techniques, and other representation styles, techniques, tools, conventions. Really there is not much of a trick to it - you just have to:

  1. use trimetric, instead of isometric, drawing, and
  2. also add in the perspective effect.

The combination of these two techniques results in drawings that:

  1. Still test the ability to perform spatial rotations "in one's head".
  2. When combined into a test, that test's psychometric properties still show reliability and construct validity.
  3. Still fulfill the valid requirement that they should be reproducible in black-and-white line-drawing form,
  4. Inflict a minimum of pain, anguish, confusion.
  5. Are attractive - the "drawings" "draw" in the sense of "drawing one in" or "drawing one's interest".

Of course the ORT-3 is still new and some of those claims - especially number 2 - are aspirational and have yet to be proved out. For now, the ORT-3 is a better test for me and my purposes. It will certainly undergo revision as experience and further study show that certain items are better than others - and this is anticipated, planned for, and supported by:

The test as I have released it, is available for use. However it is also part of a DIY (Do It Yourself) framework, amenable to rapid redesign and iteration, and freely available (openly licensed according to the license terms below) for just that use.

About copyright and Clean Rooms

A final note about provenance, originality, "defensive development", avoiding accusations of copyright infringement.

Though Bodner and Guay's freely offered permission to use their 20-item test, no artwork nor object used in the ORT-3, is a drawing or redrawing of any of their work. All ORT-3 material is original artwork consisting of original drawings of distinct, different, original objects.

The method for accomplishing the two-part goal:

  1. Come up with a test of spatial ability that does the job as well as - or in this case, putatively better than - another instrument, and
  2. avoid copyright infringement, and/or potential claims/accusations thereof,

is called "clean-room design". It's well-established in the software industry where these sorts of situations are routinely encountered - you want to replicate and/or improve on some particular system's functionality, yet you're not allowed to use the original work. The system works because of the well-established legal principle that you can't copyright an idea.*

What you do, is create a "clean room" consisting of persons with no contact with or knowledge of the original work, the function of which you want to replicate. In an iterative process you explain to them the requirements of the new system (or in the case of the ORT-3, the new test); the various applicable constraints, limitations, etc. As they produce product(s) you may instruct them that something about the particular product (or iteration) is off - some particular requirement is not met. Just for one example of how this was done in this case: "Your figures need to be either 3 or 5 units on a side." You never convey to the "clean room folk" any information about the original work. Throughout the process they remain entirely ignorant of the original(s) - in this case, none of the participants had any exposure to the Guay variants, not even a glance. (I was careful to ensure that no such materials were even in the building.) Thus it is impossible for them to have "copied" the original artwork as they have never seen it.

I will repeat this below in the acknowledgements, but my thanks to the Ad Hoc ORT-3 Clean Room team for creating original 3-dimensional objects used as basis for the drawings included in the ORT-3. More information about clean-room design is available at https://en.wikipedia.org/wiki/Clean-room_design.

*You can, in the U.S.A, legally patent a method but that is a different beast.

Conclusion, and a final note about Isometric Ambiguity

If you followed this link (Isometric Ambiguity, also given above, toward the top of this document), you may have learned that:

  1. it's a (real) thing, and
  2. nailing it down and describing just why and how it's a real problem, takes a lot of time. Even to read all that takes some time and effort.

So what is the big deal, about it? Why all the effort?

Personally I'm neither a draftsman, mechanical engineer, nor anything of that sort. I'm a supporter of talented youth who has put together some screening instruments - to support talented youth you have got to first identify them. Spatial ability (or "spatial intelligence") is an important third leg of intelligence after "the other two", but what are those other two? They variously get described like this:

Mulling over the above list, there always seem to be two other "legs of the stool" that draw all the attention, not to mention all the resources. Somehow spatial ability gets short shrift. Thus my effort to create the ORT-3, and thus my introduction to isometric ambiguity.

My own personal "take-away" from all of this is, I don't want my testing toolkit, to be contaminated by isometric ambiguity. There's no reason for it, it causes inaccuracy and negative feelings (such as rage, disdain, sadness, contempt), and there are perfectly good alternatives. Of which the ORT-3, hopefully, is one.

Resources

If you want to treat the ORT-3 as a DIY project, or use the materials in support of some other project, these files will get you started. But note:

References:

Bodner and Guay (1997)
The Purdue Visualization of Rotations Test; George M. Bodner and Roland B. Guay; The Chemical Educator, Volume 2, No. 4
DOI: 10.1007/s00897970138a

Uttal DH, Meadow NG, Tipton E, Hand LL, Alden AR, Warren C, Newcombe NS. The malleability of spatial skills: a meta-analysis of training studies. Psychol Bull. 2013 Mar;139(2):352-402. doi: 10.1037/a0028446. Epub 2012 Jun 4. PMID: 22663761.

Nora Newcombe, Thomas Shipley
Thinking About Spatial Thinking: New Typology, New Assessments
November 2015

Jianping Yue
Spatial Visualization by Realistic 3D Views
The Engineering Design Graphics Journal
Vol. 72 No. 1 (2008): Winter
Published: 2019-05-12
DOI: https://doi.org/10.18260/edgj.v72i1.12

Psychometric Properties Of The Revised Purdue Spatial Visualization Tests: Visualization Of Rotations (The Revised PSVT:R)
So Yoon Yoon
(Ph.D. dissertation)
August 2011
Purdue University Graduate School Thesis/Dissertation
https://docs.lib.purdue.edu/dissertations/AAI3480934/

Acknowledgement

Thanks to "Yoona" (So Yoon Yoon) for her permission to use her revised 30-item test for research purposes. Though it turns out that for reasons described above, I do not, nor do not intend to, use that instrument nor any portion of it, it was kind of her to get back to me.

My thanks to the Ad Hoc ORT-3 Clean Room team for creating original 3-dimensional objects used as basis for the drawings included in the ORT-3. More information about clean-room design is available at https://en.wikipedia.org/wiki/Clean-room_design.

Information: [email protected]

License

Some materials presented here freely offered by Flocabulary according to their own statement ("This vocabulary list is free and printable").

All other materials including the text of this website Copyright 2025 Leo Hesting under the Creative Commons Attribution-ShareAlike 4.0 International license. Briefly,
You are free to:
  1. Share - copy and redistribute the material in any medium or format for any purpose, even commercially.
  2. Adapt - remix, transform, and build upon the material for any purpose, even commercially.
The licensor (Leo Hesting) cannot revoke these freedoms as long as you follow the license terms.

Under the following terms:

  1. Attribution - You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  2. ShareAlike - If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
  3. No additional restrictions - You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
For more information see https://creativecommons.org/licenses/by-sa/4.0/