Virtual methods in palaeontology

So this post is a much delayed one (I'd like to blame work, but it's been more slacking on my behalf). I taught a couple of geology MSc students a few weeks back about some of the virtual/digital methods used in palaeontology, which focussed on the ones I have some experience with. Inevitably this means I will have missed some and for that I will have to return in later blogs! Any program names used here are only because I have experience, and is not to say they are the only, let alone best for the methods. I would encourage people to try out as many as they can to find out what is best for them and for pretty much every product that involves a license, there are free equivalents.

Laser Scanning/Photogrammetry

Laser scanning - As the name suggests the laser scanner uses a laser to scan the surface of an object. It works by shining a laser that reflects off of the surface and is picked up by a receiver. In the case of most palaeotological applications the scanners are based on triangulation - the receiver has a central location where reflected light hits for a known distance. Deviations from this location are caused by the object being scanned being closer or further away. For each scan a point is produced that get coalesced into a cloud that define the object. Some modern scanners also incorporate cameras that take photos of the object being scanned and overlay the photographs onto the cloud to create the entire surface/texture. Scanners are used particularly for objects where they cannot be CT scanned due to size, and only the external shape is required (as the scanner captures no internal structure) and where high detail (on the micron scale) is required. However, laser scanning can be quite time consuming and as it is based on line of site, often struggles around many complex structures e.g. the struts in skulls, which result in gaps in the model emerging unless additional scans are carried out for these areas.

Laser scanning a sauropod vertebra
http://www.theverge.com/2012/7/2/3105916/3d-printing-dinosaur-fossils-drexel-lacovara

Photogrammetry - Much like laser scanning, this method works by creating a surface of the object. However, unlike laser scanning, requires no exceptional equipment and is based only on photographs. Indeed these photographs don't even have to be digital for this method to work. A series of photographs is taken of the object, roughly equally spaced (in terms of degrees if imaging the entirety of an object) all the way around the object. This is then repeated at different angles (normally higher or lower angled than the first photo series) as many times as wanted. These photographs are put into a computer program which identifies perspective and relative landmarks (sometimes automatically, sometimes requiring user input depending on the program) and using this creates a point cloud as in laser scanning. This point cloud can have the textures of the photos overlain to create the object digitally. A good example is the Stegosaurus specimen at the NHM in London which was reconstructed digitally for some of the research using photogrammetry. A major proponent of this method is Peter Falkingham (with whom I had the privilege to share an office for almost a year) as it allows for quick and easy reconstructions of trackways and give detail far beyond what traditional ichnotaxonomists do with standard line drawings. I would recommend anyone wanting to know more about the method to read his 2012 paper about it which also provides access to some of the free software. You can even try out 123D Catch (one of the programs I have used on my computer) on your iPhone! As with laser scanning there is problems with line of site, but this can be rectified by just taking some more photos from different angles. The number of photos taken, the regularity and evenness of spacing, and the number of separate planes/angles will improve quality of the final reconstruction. I've had models work with 20-50 photos, whilst others have failed at over 100. It depends on the complexity of the structure and what you are trying to do. Experimentation is free though and is reliant only on computer power! This method has become one of the preferred ways for digitising specimens in museum due to its low cost.

Photogrammetry of an Asian elephant from Falkingham, 2012.
http://palaeo-electronica.org/content/issue1-2012technical-articles/97-264/118-264-figures#f6

Scanning and image segmentation

Scanning - Fossil scanning has been used for a long time originally as just x-ray imaging, but increasingly CT and synchotron scanning. CT and synchotron scanning involves usually radial scanning of a specimen producing (post-processing) a series of 3D x-ray scans. The biggest difference between the two methods is the energy and size of the machines. CT scanners vary from desktop size to large walk in units, but even the biggest do not compare to synchotrons. Synchotrons are particle accelerators, that specifically spin electrons around a ring at near light speed. In doing so the electrons give off x-rays, which are focussed down beam lines. Specific beam lines are set up for studying fossils (and to my knowledge 3 synchotrons in Europe currently are used study fossils - Swiss Light Source, European Synchotron Radiation Facility and Diamond Light Source).

Swiss Light Source synchotron - the giant ring at the front, but all the other buildings are associated

The choice of method is determined by size of the specimen (synchotrons don't work well with specimens beyond a few cm, unless you take many scans and merge them all together), the resolution required (a medical CT scanner will provide a lower scan quality than a microCT, which in turn is lower resolution than a synchotron), the cost and availability (synchotron time is often free and funded by research but is highly competitive, whilst access to CT scanners can run into the £100s/hr but is often easier to get time on). Beyond this there are other factors that need to be considered including length of time required to make sure good CTs are taken with no artefacts or beam hardening effects (often associated with a lack of x-rays penetrating the specimen due to the x-rays having too low energy or not enough time to allow sufficient numbers to get through). When it comes to fossils there is often a bit of trial and error involved in this process and linked to what is seen post processing and during segmentation

Piece of pliosaur jaw in Southampton CT scanner. Went on to be published in Foffa et al. 2014 (x2) with which I was involved.

Segmentation - After the resulting CT scans a process called segmentation is carried out. This is where the object of interest is isolated from the CT scans. I've endured both Avizo and Mimics with various degrees of swearing and relearning techniques as I flip between the two. They both have advantages and disadvantages depending on what you are trying to do, but I won't discuss further here (feel free to get in touch/leave a comment). The complexity of this varies from simple and often can be carried out in an automated way for single bones, to a highly intensive process that often involves many hours of someone (I have been this person many times) manually isolating the required item from the scan, be it a bone, endocast or internal anatomy.

CT scan demonstrating the god awfulness of fossils in matrix. Blue is toothy bits (mostly), although there is lots of noise in the image, whilst the red area is my deselecting pixels.

The complexity of the process is often determined by the preparation of the fossil (more matrix makes segmentation more tricky), the x-ray density of the fossil compared to the matrix (e.g. where bone is more x-ray opaque than the matrix there is clearer delineation between the two), the type of matrix (iron or pyrite rich matrix, for example, tends to make it tricky for good CTs to be made), how well scanned the item was/number of artefacts.

Using these digital preparations is becoming more common to guide the actual preparation of fossils in large museums where CT scanner access is now easy. Another thing the virtual reconstructions allow is retrodeformation (basically the undoing of damage suffered whilst the specimen is fossilised). Stephan Lautenschlager has been incredibly good at it, and I have a paper in review at the minute discussing my experience so stay tuned!

The resulting reconstructions can then be used for publication, or exported in various formats (commonly .stl or .ply) to create 3D pdfs, create 3D printouts or use in functional analyses. There are some good .stl repositories for fossils e.g. Phenome10k.org that allow of exchange and sharing of models.

Printing/3D PDFs

3D printing - 3D printing has been on the rise with the printer costs becoming far cheaper. These printers work by melting a layer of plastic or resin (SLA and ABS are the most common) and extruding this onto the print surface (not dissimilarly to standard printing). What varies however is the fact that these layers of plastic are then built up on top of each other layer by layer until the final structure is built. Nowadays a decent 3D printer with a 20x10x10 print area costs around £1000, with the cost falling year on year, and the print material is about £50 for a roll of about 1kg. However, with this decrease in cost, has come an increase in print size areas, speed, accuracy, and nowadays you can print in multiple materials at the same time (e.g. a dissolving print material to create supports that when the printout is immersed in water dissolves, leaving only the wanted 3D structure, or even some rigid and some flexible components). With 3D printing special consideration has to be given into the design of the structure as often supports need to be added (most printer programs do this automatically) to ensure that overhanging bits are printed correctly. These can be easily broken off at the end but can affect the surface quality where they attached. This is all well and good, but I hear you asking what's the point beside creating lots of pretty printouts? Well the printouts are awesome, but there are both research and outreach components to it. In terms of research, 3D printing allows for a quick and easy way of enlarging otherwise microscopic fossils into a size that can be handled and manipulated easily. In addition you can quickly send files to collaborators in other parts of the world and they can print out their own copy of the specimen in question rather than risking damaging or destroying it in transport (particularly if it is fragile). They may also be useful for recreating original specimens if it is lost, but scans exist  and when no other method (traditional casting and moulding) will work. In terms of outreach, they are incredibly useful to take these replicas to show people what you work on in terms of fossils or research that again otherwise runs the risk of being damaged whilst on show.

Let's be honest this is cool, and you kind of want one.
http://www.earthmagazine.org/article/changing-landscape-geoscientists-embrace-3-d-printing

3D pdfs - This is much like 3D printouts, except in its digital format that opens with most Adobe Readers. 3D pdfs allow for users to gain access to reconstructions, and in the highest quality ones can allow users to interact by removing components (often things like soft tissues from around skulls). Again these have the ability to be rapidly sent to collaborators worldwide, but they also allow for more detailed figures within publications to allow for better understanding of features that are being described. If you are unsure of what I mean, please do go check out the Witmer Lab's work as they are one of the main users for both scientific and outreach purposes.

Functional analyses

Beam theory - This method works on the principle of bending beams. If you assume a beam is held at one end (a cantilever beam) and a force is applied to the other it will produce tension on one side and compression on the other. Due to this, there must be a region somewhere that undergoes neither compression or tension, the neutral axis. How the material is distributed away from this determines how resistant to bending an object will be:

Shamelessly taken from my own paper on spinosaurs:  (A) When a load is applied to a beam with one fixed end (a cantilever beam), the effect of the beam is a deflection in the direction of the force. This results in the most extreme tension on one side of the beam, and the most extreme tension on the opposite side. In the middle, there is a point where there is no tension or compression, called the neutral axis. B) Two circular cross sections of equal cortical area (black). Beam theory states the solid tube (hollow circle) will have higher resistance to bending and torsion than the solid circle due to the material being distributed further from any neutral axis. DOI10.1371/journal.pone.0065295.g004

For this method to work best cross sections are needed, and these are best acquired from CT scans, although other methods work. There are many papers out there on the method (it was my first paper), but this method is falling out of favour with more complex models like FEA. It does however maintain use as a predictor of relative resistance to bending and torsion of objects and allows for gross comparisons across taxa.

FEA - See the earlier blog post I did for lots of details.

Musculoskeletal modelling - This method is one that is increasingly popular for understanding the influences on muscles and bones, and how they interact in posture and locomotion. I am still a newbie at this and am learning the method now. However, it is already extensively used within palaeontology to answer questions such as how fast could T. rex run? Effectively the structure in study (let's work with limbs here because that's easiest), is modelled. The limb then has muscles attached as informed by either muscle scarring locations or by using extant phylogenetic bracketing (finding the closest relatives and using them to help inform us of extinct life). The model can then be tweaked with muscle parameters - mass/force production/relative contribution of tendons etc etc. (again inferred from modern relatives), and then the computer can work out ideal postures, how fast the limb (and animal) can move, how big muscles need to be to move at certain speeds, what order muscles are likely to activate in to allow biologically reasonable movements.

T. rex model with all of the muscles attached to the limb and pelvis, from Hutchinson et al., 2005

Convex hulls/Body mass estimates - I will not attempt to explain this method in much detail, beyond saying that the method takes the original skeleton (or limb or whatever biological structure) and attempts to wrap surfaces over it which may infer the extent of the soft-tissues overlaying the structure. I direct all interested parties to Pete's blog which has links to papers describing the method, and walks you through how to do it yourself.

Stegosaurus convex hulls. From Brassey et al., 2015

So that wraps (pun definitely intended) up some of the methods we use and why we use them when it comes to palaeontology on computers. Most of these methods are less than 20 years old. Just imagine where we will be in the next 20!