The IRMCS group 1990 - 2000
At the MTMM conference in Amsterdam Frank Andrews and I realized that we would never be able to study the quality of survey questions, if we did not start a research group with representatives of different countries, where different languages are spoken. If in all these countries MTMM experiments were done, we would get basic information about the quality of questions in different languages for different topics. This was a very interesting idea, therefore I looked at my files for possible candidates from different countries to participate in such a research group. I invited besides Frank Andrews (US) the following people , most of them were prior students from Essex or I had met them at one of the conferences: Anuska Ferligoj (Slovenia), Anna Andreenkova (USSR), Juan Manuel Batista (Spain), Jaak Billiet and Hilde de Wit (Belgium), Brendan Bunting (Northern Ireland), Richard Költringer (Austria), Peter Schmidt and Dagmar Krebs(Germany) , Arne Kolstadt (Norway), Keith Roe (Sweden). Later Akos Münich (Hungary) joined the group just as Irena Bartova and Jan Rehak (Czechoslovakia) and students of the leading researchers like Annette Scherpenzeel (The Netherlands), Tina Hlebec (Slovenia), Germa Coenders (Spain). Below is a foto of the people present at the meeting in Prague, one of the few occasions that we made a offical foto.
The first meeting
Given my good contacts with Anuska Ferligoj the suggestion was made that the first meeting would take place in Slovenia and she organized it in the beautiful place Bled. There along the lake we discussed the MTMM approach and the idea that one would get estimates of the quality of questions and that one could use this information to correct the observed correlations for measurement errors in the questions used. We planned to start with a pilot study where all countries would do the same study, using quality of life questions which would be the same in all countries so that we could also compare differences between the countries with respect to the quality of questions and correlations between the variables. This plan was agreed and we made an appointment for a next meeting.
Given my good contacts with Anuska Ferligoj the suggestion was made that the first meeting would take place in Slovenia and she organized it in the beautiful place Bled. There along the lake we discussed the MTMM approach and the idea that one would get estimates of the quality of questions and that one could use this information to correct the observed correlations for measurement errors in the questions used. We planned to start with a pilot study where all countries would do the same study, using quality of life questions which would be the same in all countries so that we could also compare differences between the countries with respect to the quality of questions and correlations between the variables. This plan was agreed and we made an appointment for a next meeting.
Several other meetings in Slovenia
Bled was a beautiful place but not the cheapest in the world. We had no funding for these activities, therefore all participants were supposed to pay their own travel and hotel costs. This meant that several meetings took place at another nice place in Slovenia called Predvor but also in Ljubljana and once in Piran.
Bled was a beautiful place but not the cheapest in the world. We had no funding for these activities, therefore all participants were supposed to pay their own travel and hotel costs. This meant that several meetings took place at another nice place in Slovenia called Predvor but also in Ljubljana and once in Piran.
Meetings outside Slovenia
It will be clear that we could not always ask Anuska to organize the meetings for us. Therefore, later the meetings moved to different places. We went to Prag, Amsterdam, Mannheim and Debrecen. Below I show some typical pictures of our meetings in the different towns. The picture from Prague was the offcial photo you have seen above.
It will be clear that we could not always ask Anuska to organize the meetings for us. Therefore, later the meetings moved to different places. We went to Prag, Amsterdam, Mannheim and Debrecen. Below I show some typical pictures of our meetings in the different towns. The picture from Prague was the offcial photo you have seen above.
The last meeting in St Petersburg
Finally we were also invited to a meeting in St. Peterburg organized by Wladimir Andreenkov, his wife and their daughter, Anna, who participated in most of our meetings. This was a special event in different ways. Most of us had not been in Russia and in St Petersburg. Our hosts organized nice sightseeing in the town and surroundings. Parts of the town were old and run down but others were of course beautifully renovated like the Hermitage.
Finally we were also invited to a meeting in St. Peterburg organized by Wladimir Andreenkov, his wife and their daughter, Anna, who participated in most of our meetings. This was a special event in different ways. Most of us had not been in Russia and in St Petersburg. Our hosts organized nice sightseeing in the town and surroundings. Parts of the town were old and run down but others were of course beautifully renovated like the Hermitage.
Did all these meetings lead to something?
You may think that we met only for fun each year at a different place. That was true, but we had also long discussions about methodological issues we were involved in. About the methodological issues that we discussed, a book was produced.
In all 16 countries represented in the group, the study with respect to quality of life questions was done. The data were analyzed and also summarized in a reader with a description of the quality of life results in the different countries and an analysis about the factors that determine the quality of life in each country.
I have to admit that at the end I was a bit disappointed about the results of the group because only in the Netherlands, Belgium and Austria several MTMM studies had been done which could be used to evaluate the quality of different question forms.
You may think that we met only for fun each year at a different place. That was true, but we had also long discussions about methodological issues we were involved in. About the methodological issues that we discussed, a book was produced.
In all 16 countries represented in the group, the study with respect to quality of life questions was done. The data were analyzed and also summarized in a reader with a description of the quality of life results in the different countries and an analysis about the factors that determine the quality of life in each country.
I have to admit that at the end I was a bit disappointed about the results of the group because only in the Netherlands, Belgium and Austria several MTMM studies had been done which could be used to evaluate the quality of different question forms.
A word about Frank Andrews
Let me add here a few words about Frank Andrews. His research and his paper of 1984 was the basic stimulus for all this work. The work we did was more a continuation of what he started with some minor improvements. Unfortunately he has not seen anymore what was the consequence of his work because he died suddenly in 1995. With his death we lost a great inspirator of our research and an amiable friend.
Let me add here a few words about Frank Andrews. His research and his paper of 1984 was the basic stimulus for all this work. The work we did was more a continuation of what he started with some minor improvements. Unfortunately he has not seen anymore what was the consequence of his work because he died suddenly in 1995. With his death we lost a great inspirator of our research and an amiable friend.
This work had its consequences
For Austria Richard Költriger did a meta- analysis of his studies and made a book about survey research and the quality of questions in German. Annette Scherpenzeel used the Dutch and Belgium data to do the same for Dutch questions and published her results in her dissertation.
Later I combined the data of the USA, Austria and the Netherlands for a meta-analysis to determine the effect of question characteristics on the quality of the questions in these three languages.
Then I also made the first program SQP to predict the quality of questions on the basis of the question characteristics. The limitation of the program was of course that it could only predict the quality of questions in three languages. This work was useful because it showed us what could be done and what the result would be.
For Austria Richard Költriger did a meta- analysis of his studies and made a book about survey research and the quality of questions in German. Annette Scherpenzeel used the Dutch and Belgium data to do the same for Dutch questions and published her results in her dissertation.
Later I combined the data of the USA, Austria and the Netherlands for a meta-analysis to determine the effect of question characteristics on the quality of the questions in these three languages.
Then I also made the first program SQP to predict the quality of questions on the basis of the question characteristics. The limitation of the program was of course that it could only predict the quality of questions in three languages. This work was useful because it showed us what could be done and what the result would be.
A very different story
In St Petersburg the organization was partly done by a colleague of Anna Andreenkova who is on the picture below. At one occasion she told me a story about her last holiday which is at least as unusual as the story of Gravilov, I have told before. She said that she went with some friends on a raft down one of the larger rivers in Russia. If I remember it correctly they were with 4 people, two men and two women. They had constructed the raft themselves and it was quite large to carry them and a hut. When the raft was ready and they could handle it well, they started to go down the river. All went well till they came to a point where the river became smaller close to a village. There they could not go on because the raft was too broad. That was unexpected, they decided to adjust the raft but that took some days. Due to this delay they needed to buy extra food in the village. There they detected that in this area money could not be used. Here it was still so that transactions were done on the basis of exchange of goods while the basic needs were covered by the government of the region.
This happened in the year 2000: people make a plan to go down a river and end up at a place where people were not yet part of the money economy while on the other hand in the same country, ten years before Gravilov developed his own computers for computer assisted interviewing. The differences can´t be larger.
In St Petersburg the organization was partly done by a colleague of Anna Andreenkova who is on the picture below. At one occasion she told me a story about her last holiday which is at least as unusual as the story of Gravilov, I have told before. She said that she went with some friends on a raft down one of the larger rivers in Russia. If I remember it correctly they were with 4 people, two men and two women. They had constructed the raft themselves and it was quite large to carry them and a hut. When the raft was ready and they could handle it well, they started to go down the river. All went well till they came to a point where the river became smaller close to a village. There they could not go on because the raft was too broad. That was unexpected, they decided to adjust the raft but that took some days. Due to this delay they needed to buy extra food in the village. There they detected that in this area money could not be used. Here it was still so that transactions were done on the basis of exchange of goods while the basic needs were covered by the government of the region.
This happened in the year 2000: people make a plan to go down a river and end up at a place where people were not yet part of the money economy while on the other hand in the same country, ten years before Gravilov developed his own computers for computer assisted interviewing. The differences can´t be larger.