Research topics:

Bidirectional Texture Function Analysis and Modelling

The ultimate goal of computer graphics is realtime and accurate simulation of realword objects appearance in virtual scenes. Bidirectional Texture Function (BTF) is currently one of the most accurate and affordable way of genuine materials appearance representation.

 What is BTF?
The BTF describes variations of material surface texture appearance due to varying illumination and viewing directions. This 6D function is measured by taking a large number of still images for thousands of combinations of illumination and viewing directions [DEMO].

 What is BTF useful for?
Application where accurate + interactive + realistic material appearance is a must:
- Interior design in architecture, automotice industry etc.
- Visual safety simulation in automotive, airspace industry etc.
- Digital preservation of culture heritage objects.

 Why we cannot use the measured BTF directly?
BTF measurement results in massive datasets (~GBs/material) prohibiting any interactive application and hence compressed representation providing fast reconstruction and modelling of these huge texture data is inevitable.

Our recent IEEE TPAMI survey article on this topic is available online. BTF compression and modelling was also a main topic of my dissertation thesis Colour Rough Textures Modelling. See below my past research projects on BTF related topics.

 BTF Compression Based on Multi-Level Vector Quantization

In this paper a novel BTF compression model is proposed. The model resamples input BTF data into a parametrization, allowing decomposition of individual view and illumination dependent texels into a set of multidimensional conditional probability density functions. These functions are compressed in turn using a novel multi-level vector quantization algorithm. The result of this algorithm is a set of index and scale code-books for individual dimensions. BTF reconstruction from the model is then based on fast chained indexing into the nested stored code-books. In the proposed model, luminance and chromaticity are treated separately to achieve further compression. The proposed model achieves low distortion and compression ratios 1:233-1:2040, depending on BTF sample variability. These results compare well with several other BTF compression methods with predefined compression ratios, usually smaller than 1:200. We carried out a psychophysical experiment comparing our method with LPCA method. BTF synthesis from the model was implemented on a standard GPU, yielded interactive framerates. The proposed method allows the fast importance sampling required by eye-path tracing algorithms in image synthesis.

Psychophysics of Uniform Sampling and Eye-Tracking of BTFs

We employ perceptually-based methods to allow more efficient handling of BTF data. In the first step we analyse different uniform resampling by means of a psychophysical study with eleven subjects, comparing original data with rendering of a uniformly resampled version over the hemisphere of illumination and view dependent textural measurements. We have found that down-sampling in view and illumination azimuthal angles is less apparent than in elevation angles and that illumination directions can be down-sampled more than view directions without loss of visual accuracy. In the second step we analysed subjects gaze fixation during the experiment. The gaze analysis confirmed resuls from the experiment and revealed that subjects were fixating at locations aligned with direction of main gradient in rendered stimuli. As this gradient was mostly aligned with illumination gradient we conclude that subjects were observing materials mainly in direction of illumination gradient. Our results provide interesting insights in human perception of real materials and show promising consequences for development of more efficient compression and rendering algorithms using these kind of massive data.

A Psychophysically Validated Metric for BTF Reduction  

The key to providing efficient compression of BTF is the decision as to how much of the data should be preserved. We use psychophysical experiments to show that this decision depends critically upon the material concerned. Furthermore, we develop a BTF derived metric that enables us to automatically set a material's compression parameters in such a way as to provide users with a predefined perceptual quality. We investigate the correlation of three different BTF metrics with psychophysically derived data. Eight materials were presented to eleven naive observers who were asked to judge the perceived quality of BTF renderings as the amount of preserved data was varied. The metric showing the highest correlation with the thresholds set by the observers was the mean variance of individual BTF images. This metric was then used to automatically determine the material-specific compression parameters used in a vector quantisation scheme. The results were successfully validated in an experiment with six additional materials and eighteen observers. We show that using the psychophysically reduced BTF data significantly improves performance of a PCA-based compression method. On average, we were able to increase the compression ratios, and decrease processing times, by a factor of four without any differences being perceived.

A novel fast probabilistic model-based algorithm for realistic BTF modelling allowing such an efficient compression. Its ultimate aim is to create a visual impression of the same material without a pixel-wise correspondence to the original measurements. The analytical step of the algorithm starts with the BTF space segmentation and range map estimation of the BTF surface followed by the spectral and spatial factorisation of selected sub-space multispectral texture images. Single monospectral band-limited factors are independently modelled by their dedicated causal autoregressive models. During rendering the corresponding sub-space images of arbitrary size are synthesised and both multispectral and range information is combined in a bump mapping filter of the rendering hardware according to view and illumination directions. The presented model offers huge BTF compression ratio unattainable by any alternative sampling-based BTF synthesis method. Simultaneously this model can be used to reconstruct missing parts of the BTF measurement space.

This model is based on modified and extended Lafortune reflectance model computed per each texel. The extension consist in adding a few spectral parameters for each BTF image which are linearly estimated according to original data in second estimation step. A model parameters are computed for every view reflectance field contained in original BFT data. The final memory BTF data storage demands are with using of this technique reduced in ratio 1:15 when the synthetised images are almost indiscernible from originals. The method is universal, robust and easily implementable in a graphical hardware.

A novel efficient probabilistic model-based method for multispectral BTF texture compression is capable of seamless BTF space enlargement. The analytical step of the algorithm starts with BTF texture surface estimation followed by the spatial factorization of an input multispectral texture image. Single band-limited factors are independently modelled by their dedicated 3D causal autoregressive models. We estimate an optimal contextual neighbourhood and parameters for each model. Finally the synthesized multiresolution multispectral texture pyramid is collapsed into the required size fine resolution synthetic smooth texture. Resulting BTF is combined in a displacement map filter of the rendering hardware using both multispectral and range information, respectively. The presented model offers immense BTF texture compression ratio which cannot be achieved by any other sampling-based BTF texture synthesis method.

This work presents a fast model-based algorithm for realistic multispectral BTF texture model. The algorithm starts with surface range-map estimation from one texture image based on shape from shading technique. The estimated range-map is finally combined with probabilistic smooth synthetic texture. Synthetic BTF image is rendered according surface range-map for required view and illumination angle. The presented model offers huge BTF texture compression ratio which cannot be achieved by any other sampling-based BTF texture synthesis method (only one range map and about hundred of real numbers have to be stored). It is possible to generate BTF texture of arbitrary size in case when also the range map is modelled.

Dynamic Textures and Video Sequences Analysis and Modelling

Dynamic textures and video sequences represent challenging 3D datasets comprising mutually dependent spatial and temporal information. Some of effects present in such a data are typical and may be analysed and exploited for variety of image processing tasks. See below my past research projects on this topic.

Automatic Temporal Segmentation of Videosequences

With significantly increasing number of archived movie sequences raises a need of their automatic indexation and annotation. Robust and fast temporal segmentation of video sequences is one of the challenging research topics in this area. In this paper we propose a new temporal segmentation method of the video sequences based on PCA approach. Contrary to standard approaches based on histogram or motion field analysis the proposed method does not require any such a complex analysis. The method starts with sparse greyscale sampling and eigen-analysis of input sequence. A sum of absolute derivatives of temporal mixing coefficients of main eigen-images is then used as cuts detection feature, while dissolve transitions are detected by means of coefficients' specific behaviour. The functionality of the method was successfully tested on number of sequences ranging from artificial set of similar dynamic textures to professional documentary movies. Although, the results may not be unexpected, we believe that proposed method provides novel, very fast and reliable way of movie cuts detection.

Textural appearance of many real word materials is not static but shows progress in time. If such a progress is spatially and temporally homogeneous these materials can be represented by means of dynamic texture (DT). DT modelling is a challenging problem which can add new quality into computer graphics applications. We propose a novel hybrid method for colour DTs modelling. The method is based on eigen-analysis of DT images and subsequent preprocessing and modelling of temporal interpolation eigen-coefficients using a causal auto-regressive model. The proposed method shows good performance for most of tested DTs, which depends mainly on properties of the original sequence. Moreover, this method compresses significantly the original data and enables extremely fast synthesis of artificial sequence, which can be easily performed by the means of contemporary graphics hardware.

This work presents a new type of scratch removal algorithm based on a causal adaptive multidimensional multitemporal prediction. The predictor use available information from the neighbourhood of a missing multispectral pixel due to spectral, temporal and spatial correlation of video data but not any information from the failed pixel itself. The model assumes white Gaussian noise in each spectral layer, but layers can be mutually correlated. A significant improvement of the 3D model performance is obtained if the temporal information is included, i.e., using the 3.5D causal AR model. Such information is natural to obtain from previous or/and following frame(s) for which we know all necessary data, due to high between-frame temporal correlation. Thanks to this we can treat data from different frames (specified by the contextual neighbourhood) in the same way, so we attach to each data information about its shift according to predicted pixel placement. The contextual neighbourhood has to be causal (in the reconstructed frame lattice subspace) . It means that the predictor can use only data from the model history. Then if we assume normal-Wishart parameter prior the predictor has analytical (not iterative) solution.
Image sequence restoration was also the main topic of my diploma thesis Colour Movies Scratch Restoration.

3D Data Measurement - Phlegmatic Dragon Model

Download free Dragon 3D model for your research
(official model of Eurographics 2007)