Let's get down to the nitty gritty! What is needed to actually DO a project like this? What advice can you share with others new to this field?
- How do you measure the impact of the project?
- What metrics and guidelines are used to measure the successful use of technology for transparency?
- What advice do you have for organizations looking for funding for this kind of work?
- How do you know if a project is sustainable?
Share your experiences, thoughts, questions and ideas by adding a new comment below or replying to an existing comment!
When we first started almost no one was building applications for transparency in Latin America. Our first project in 2009, Vota Inteligente, designed a program that was in coordination with the Chilean presidential elections. The program called “Media Naranja” (Your political other half) contained a series of questions that a person could answer and then matched them to the candidate that they should vote for. It also monitored all of the webpages and social media sites, and platforms of the candidates. This site was widely used, and was essential to the start of our organization because of its timing, the communications effort by the organization, and most importantly, there was a need! After the elections, we wanted to make this project sustainable but couldn’t have possibly maintained the site the way it was, as matching voters with candidates, because the elections were over and it would have been irrelevant. However, we adapted Vota Inteligente to become a permanent interactive legislative monitoring platform that allowed citizens to inform themselves about the actions of Parliament, something that hadn’t previously existed in Chile, and is one of our biggest projects to date.
In measuring the success of projects, we are constantly working to improve that, but in the past it has been through through the number of tweets/retweets, social media followers, the number of unique monthly users, the number of stories published or recorded by traditional media, the number of organizations involved in the use of our applications and websites, and evaluate the reaction of government authorities and public institutions to the applications, websites and citizen action through any of the Ciudadano Inteligente platforms. What do other people do to measure success?
Thank you for sharing this, Liz! It's helpful to hear that your organization started with one project that was focused on one electoral event. It sounds like Vota Inteligente saw the need and really designed something that was useful. I would be curious to know if you carried out a needs-assessment or something like it to know what the needs and requirements of civil society organizations were around access to information - or if your organization knew exactly what those needs were because they had been working on transparency issues for so long.
Also, I am really interested in learning more about how Vota Inteligente:
Do funders require that you measure these aspects or have you found that it is useful for understanding your own impact? Thanks, Liz! Others - please share how you are measuring the impact of your projects!
As mentioned before, we are still developing the ways in which we evaluate the success of our projects, and are open to hearing about other peoples’ experiences and suggestions!! We closely monitor the number of visits of our sites. Through google analytics, we are able to obtain information on the frequency and number of users of our applications. We also very closely monitor our social media accounts, including the number of followers on our accounts, the tweets and posts that are written. The measurement of the reaction of outside authorities is shown through the number and quality of media appearances and coverage, as well as the accomplishments made with various members of Parliament. This could mean opening data, policy reform, or changes in regulations.
The project, Acceso Inteligente, lists the total number of solicitations (we have 586 in 5 months) that have been sent to the various ministries that we are connected with. We not only measure our site in number of information requests, but also in the number of users, since it is also supposed to be used as a platform for citizens to seek already requested and responded to information. This means that measurement of the success of our application doesn’t solely rely on the pure number of information requests, but also the ability of the application to get responses from the government and also the interest that people take in the request and response. One information request, and response, can generate much interest among the population. For example, a citizen requested information on how much money the Chilean government spends on tear gas per year. Over 5,000 people viewed this one information request, showing that there was a clear need and interest in obtaining this information.
We also monitor the number of information requests made through our application and compare those which are made in the general universe of government website information requests.
We have found the knowledge that we have of our projects has been beneficial for both funders as well as useful for our own impact. Seeking information on our users helps us obtain more funding, have more knowledge about our users, and understand how and where to cater our projects.
Hi all! I wanted to ask this question to everyone - where do you start? - and pull together a few of the ideas that were shared already in other discussion threads. Also, this is an attempt to reply to Khan Agha Dawoodzai's comment in the first discussion thread:
Two ideas shared in this dialogue that stood out to me regarding places for practitioners to start when considering the development of a tech tool for transparency are:
1. Convene meetings that bring together practitioners interested in transparency to identify the needs and issues
Maya added a comment to the first discussion thread in which she mentions that in the very beginning stages of her transparency project, she brought together interested practitioners to identify their information/data needs:
To me, this seems like a really great place to start before desigining a tech tool. Not only with these practitioners and interested groups help you identify the needs and case uses for your tool, but you are also getting them on board from very beginning. The idea is that once the tools is ready, you will be able to go back to this same group and already have an interested and engaged set of users. Do you agree? It would be great to hear of more examples of this. I'm sure that others have used surveys, Skype calls, etc to collect this kind of information from the people that will be using your tech tool.
2. Ensure that your own organization is putting good transparency policies into place
In the first discussion thread, Mike noted that it is important for us not to forget that we also need to be transparent in our work:
It seems to me like this would be another great place to start before developing your tech tool. It is important to identify the ways in which your own group of practitioners will be transparency about its intentions, etc. How have you promoted transparency within your own work/project/organization?
Indeed, self-transparency on part of practicioners (those not at personal risk of course) and NGOs is a good place to start.
Questions about metrics and funding are endemic to all organized activity. I'm curious as to whether transparency projects, as a large class, have anything specific that sets them apart from other organized activity, or let's say from other technology-oriented civil society activity, to narrow just a bit. I can't think of anything off the top of my head. Follow no-brainer best practices. If you do that, you're way ahead of most projects. :-)
From the experience of the research done by the Technology for transparency network, I will say that the ingredients vary depending on the project:
- There are projects requiring little investment but lots of community support, such as the Elections Monitors.
- There are projects requiring a high degree of expertise to manage large databases and a lot of technical maintance, but where community is not at the core, it can survive with little once it is launched, such as the case of Poder Ciudadano Database on Campaign spending, monitoring Money in Politics.
- There are projects practically owned and ran by the community or have the community as essential part of it, such as Reclamos.cl in Chile and Quien paga Manda in Costa Rica but also require a lot of effort from the side of community managers.
So to answer the question, success will depend on placing the right assets for the aims we want and having clear objectives from the very beginning.
I agree very much with your points. What you need to keep the project alive can be very different and should be thought of already during the development of the project. Overall I think it's crucial to think of the efforts needed to keep a data based project up to date for a longer time. Otherwise there is the danger of creating nice project that end up as data ruins.
I think that while projects have their ups and downs, one must differentiate those projects who need to be up all the time from those projects who are just for the occassion. While a site monitoring the parliament, such as Curul 501 in Mexico and Congreso Visible in Colombia, requires constant work and it gets better and richer with time, some projects are for one time and even if the site do not remain online, the community is active and now how to activate a network when they need it, like the #InternetNecesario case.
My main point is that it does not matter if the project last forever, as long as it last long enough for the purposes it was created.
I agree, the criteria to measure success differs depending on the goals and scope. But also, projects are a learning process, sometimes you need to adjust your goals, make them more realistic keeping in mind the short, mid and long level of success.
Definitely, Camila! It would be great to hear some examples of short, mid and long term goals that you and others have identified for past projects. Thanks!
Sure, Kristin
For instance, with Todos somos dateros the main goals are:
Short-term:
-Citizens participating in the online platform: number of dateos (problem+solution) registered, new users joining the platform, number of comments and votes.
-Mentions on the social networks and traditional media as well (radio was accessible for us)
This was successfully accomplished, of course it is always possible to improve.
Mid term:
Authorities replying back and giving feedback on the dateos (when problems are going to be solved, when not possible, why it´s not possible, etc).
This goal was partly accomplished, as I mention on the first thread for the Cyclist section we have launched an application called the Inbox datero, this is already being used, so citizens are getting feedback directly from authorities. But for the Metropolitano (bus) section, we have to find a different strategy: the authorities feedback would be on the reports instead of replying back to the dateos.
Long-term:
Authorities implementing the solutions as proposed in the dateos (after a deliberation process and selection of the most urgent/relevant dateos)
Additional to this quantitative results, the project also contributes to understanding the complexity of urban mobility and public transport and encourage a culture of dialogue, participation and appropiation between citizens and authorities.