EVALUATION REPORT
An Evaluation of the Bureau of Oceans and International Environmental and Scientific Affairs
Lacey Act Amendment Foreign Assistance Programming
July 2016
This publication was produced at the request of the United States Department of State. It was prepared independently by Dr. Su Chuen Foo, Elizabeth Naro, Julia Benjamin, and Dr. Joseph Weinstock for Social Impact.
AN EVALUATION OF THE BUREAU OF OCEANS AND INTERNATIONAL ENVIRONMENTAL AND SCIENTIFIC AFFAIRS LACEY ACT AMENDMENT FOREIGN ASSISTANCE PROGRAMMING
July 28, 2016
IDIQ Contract Number: S-AQMMA-12-D-0086
Technical and Advisory Services for Program Evaluation Requirements
Task Order Number: S-AQM-MA-15-F-4092
COVER PHOTO
Forest Product Lab in Madison, Wisconsin (Photo Credit: Elizabeth Naro)
DISCLAIMER
The authors’ views expressed in this publication do not necessarily reflect the views of the United States Department of State or the United States Government.
EVALUATION SUMMARY
An Evaluation of the Bureau of Oceans and International Environmental and Scientific Affairs Lacey Act Amendment Foreign Assistance Programming
The evaluation was performed between September 29, 2015 and July 30, 2016. The final report was submitted on July 28, 2016.
Purpose of the Evaluation and Questions Addressed
Social Impact, Inc. (SI), an independent evaluation firm, was hired by the U.S. Department of State (DOS) Bureau of Oceans and International Environmental and Scientific Affairs (OES) to evaluate the activities conducted so far to improve implementation and enforcement of the 2008 Lacey Act Amendment. The purpose of this performance evaluation was to determine to what extent OES projects have achieved or are achieving the intended outcomes, to identify strengths and weaknesses in the design and implementation of these projects, and to provide recommendations to improve program effectiveness and outcomes. The primary users of this evaluation are OES and its implementers; Department of Justice (DOJ), The Nature Conservancy (TNC), and the United States Forest Service (USFS), but findings, conclusions, and recommendations will also inform discussions among other U.S. Government (USG) agencies, non-governmental organization (NGO) community, and industry partners.
The evaluation questions are:
1. What are the outcomes (results) of the different activities of the Lacey Act Amendment programs? What did they achieve or not achieve?
2. Were the programs appropriate to the needs and conditions of the host countries? If not, why not?
3. Which program activities have been the most and least successful, and in what contexts (geographic, focus area, etc.)? What factors and forces affect their differential progress?
4. How do OES program activities fit within broader USG support for international activities and domestic actions related to implementation of the Lacey Act Amendment? How could OES programs be better designed or positioned to leverage other USG efforts?
5. Based on past progress and on consultations, what additional partners, topics, or emerging opportunities should the State Department consider for future funding?
6. What new activities and programs do NGO partners suggest for USG agencies and OES to improve international programming in support of the Lacey Act Amendment?
Program Background. The Lacey Act of 1900, known simply as the Lacey Act, was passed by Congress on May 25, 1900. The Lacey Act was the first United States’ (U.S.) law, and one of the earliest anywhere in the world, to protect flora and fauna through civil and criminal penalties for those who violate the rules and regulations. The Lacey Act was amended in 2008, expanding coverage to a broad range of plants and plant products. This includes globally protected plants, such as those on the CITES[1] (Convention on International Trade in Endangered Species) list. More importantly, it covers plants that are not rare or endangered but have been harvested in violation of local or national laws. Enforcement of the Lacey Act Amendment has often been on internationally traded timber and wood products imported into the U.S.
With the passage of the Lacey Act Amendment, Congress mandated a $2 million annual set-aside allocation for implementation. Slightly more than half of the set-aside funds have been channeled through USAID, with the balance going to the OES Office of Conservation and Water (ECW) in DOS. With approximately $1 million annual set-aside funds for implementation of the Lacey Act Amendment, OES has focused on three intervention areas:
1. Strengthening forest law enforcement
2. Promoting stakeholder involvement in policy making and forest management
3. Supporting the development of science-based tools to assist in identifying legal forest products
The Lacey Act Amendment programs implemented by OES are the subject of this evaluation. Between Fiscal Year (FY) 2010 and FY2015, OES awarded 11 awards to its implementing partners. OES has partnered with the Department of Justice (DOJ) Environmental and Natural Resources Division (ENRD) to implement activities under the first intervention area. The Nature Conservancy (TNC) is OES’ main implementing partner for the second intervention area, and the United States Forest Service (USFS) is the main implementing partner for science-based tools development.
Methodology
In order to answer the evaluation questions, SI conducted a mixed-methods evaluation using document review, key informant interviews (KIIs), group interviews (GIs), and an online survey across three distinct phases of data collection activities. Data collection occurred between January and March 2016[2] and took place in Washington, DC; Bogota, Colombia; and Madison, Wisconsin to inform the findings, conclusions, and recommendations. A total of 26 KIIs and 6 GIs were conducted.
The Evaluation Team anticipated certain challenges that would arise from evaluating the broad range of Lacey Act Amendment programs funded by OES. In particular, the team anticipated the potential for recall, response, and selection biases to affect the data. Moreover, the broad scope of OES’ Lacey Act Amendment programming and limited evaluation timeframe and budget restricts the data and analysis that the team was able to do. OES activities are also ongoing, hence, limits the extent to which the team can provide summative findings and conclusions of OES programming.
Key Findings
Evaluation Question 1. Key informants shared that the training workshops that DOJ has conducted across Russia, Brazil, Peru, Colombia, and Cameroon have helped increase awareness and knowledge of the Lacey Act Amendment and how it can facilitate host countries’ efforts to prosecute cases against illegal logging. Through these trainings, meetings, presentations at international conferences, and other outreach activities, DOJ has been able to form relationships and develop trust with key law enforcement officials in host countries. These relationships have paved the way for an open exchange of information between country counterparts to help ongoing investigations against illegal logging, including the prosecution against Lumber Liquidators for trading of illegally harvested wood from Far East Russia, resulting in fines and penalties of over $13 million.
In Indonesia, TNC has provided technical support and capacity development to the government in developing and implementing the country’s timber verification process (SVLK), including to facilitate adoption of SVLK by smallholder wood processing operations and village producers by resolving implementation difficulties unique to these groups. Interviewees reported that there has been a marked shift in the Chinese government’s attitude on this issue, from initial suspicion over the U.S. government’s intentions behind the Lacey Act Amendment, to initial efforts to engage in illegal logging issues specific to the country. Furthermore, interviewees spoke highly about the Xylotron, a fully functioning prototype of a hand-held device and accompanying software to identify wood that has been developed and will be piloted in Brazil. The scientific image reference collection now contains over 15,000 images corresponding to 701 wood species in 172 genera within 60 families representing wood in Central America and Brazil.
While interviewees mentioned the increase in number of cases brought forward for prosecution in host countries, the Evaluation Team has not been able to objectively validate this observation. Furthermore, there is limited information to directly assess the change in host country counterparts’ increase in capacity to investigate and prosecute cases against illegal logging. While progress has been made in China, project documents and interviews suggested that there is still work to be done to bring the capacity of eastern coastal timber industry associations—critical players identified by TNC in addressing the issue of illegal timber in China—to the level needed to actively remove illegally sourced timber from their supply chains. For the Forest Products Laboratory (FPL), work remains to further develop the scientific image reference collection, including to incorporate a greater number of wood species in Central America; to correctly identify the botanical scale and scope of how to develop accurate models, how to extract features from the images in a computationally efficient manner; and to identify the underlying statistics of classifying wood. With respect to DNA species identification work, there has also been a delay in project activities as a result of seasonal flooding in the Amazon and delay in Peruvian counterparts’ field schedules.
Evaluation Question 2. Overall, key informants felt that OES projects do address the needs and conditions of the host countries, with variations in the extent of fit across countries. A major factor that attests to the appropriateness of Lacey Act Amendment activities to host country needs and conditions is the increasing realization of the severity of the situation and consequences to forest sustainability if nothing is done. Findings suggest potential host country conditions and external factors that affect the extent to which Lacey Act Amendment projects achieve their results, such as high-level support or lack thereof from host country governments, conflicting agendas among host country agencies, lack of resources, and corruption affecting law enforcement activities.
Evaluation Question 3. An early success across OES-funded Lacey Act Amendment activities is the work by FPL on the XyloTron and its corresponding scientific image reference collection. While still in prototype stage, the XyloTron has demonstrated its potential to improve enforcement and compliance with the Lacey Act Amendment across international and U.S. borders. Key informants noted that a key factor in XyloTron’s early success can be attributed to the scientists’ ability to provide the right solution to the right problem in the right country context. The presence of Dr. Sandra Florsheim’s lab and macroscopic wood identification activities indicates the need and relevance for this effort in Brazil, and likely beyond. Implementing partners cited the withdrawal of activities in Far East Russia where they had been yielding results as disappointing.
Evaluation Question 4. Much of OES programming occurs in parallel with each other. DOJ activities have some overlap with other USG agency efforts, in particular with efforts by the Bureau of International Narcotics and Law Enforcement Affairs (INL). Although the content is presented in slightly different contexts, participants often will hear repeated information in these workshops. Aside from DOJ and INL, Interpol (the International Criminal Police Organization) conducted training programs to promote the awareness and action against illegal logging. An interviewee noted that DOJ and Interpol conduct trainings within a few weeks of each other without knowledge of the other’s activities. At the same time, DOJ has coordinated with USFS scientists to demonstrate the machine vision technology at trainings in Brazil where the Fish and Wildlife Service (FWS) and DOJ have also sent presenters and participants. Beyond coordination, OES is currently co-funding USFS’ wood identification technology development with USAID and USFS International Programs. OES provided bridge funding to TNC in 2009 to support its Responsible Asia Forestry and Trade (RAFT) program when funding from USAID/RDMA ended. Since then, OES has continued co-funding the RAFT program with the Australian government.
Evaluation Question 5. Development of science-based tools in support of the Lacey Act Amendment is one of the most underfunded aspects of OES’ programming, yet it is widely recommended for further funding by actors at all levels and across all sectors. Many interviewees highlighted the lack of resources among host country law enforcement counterparts to conduct inspections and investigations, for example, transportation such as cars and gasoline and inspection materials, which are important to increase prosecutions. Interviewees also made repeated comments about the number of interagency agreements (IAAs) that OES funds compared to the magnitude of OES’ funding. Interviewees suggested that OES consider longer-term funding of fewer projects, rather than short-term funding of multiple projects. Increased funding of fewer programs or activities would allow those programs to develop more rapidly, with more tangible and significant outcomes.
Evaluation Question 6. Overall, most NGOs advised that OES could be more strategic in country selection for program implementation. NGO representatives suggested that OES shift funding to support longer-term projects. Ideally projects should work with the same beneficiaries consistently, over longer periods of time in the same country, to ensure that projects are achieving their outcomes. Furthermore, NGOs voiced the importance of leveraging existing relationships that they already have in host countries to facilitate and catalyze OES’ Lacey Act Amendment programming.
Conclusions
Evaluation Question 1. OES’ Lacey Act Amendment programming has witnessed significant progress and achievement of certain outcomes across implementing partners and host countries. However, given the nature of information collected and outcomes of the implementing partners, the team was unable to form conclusions for the achievement of certain outcomes. Some project activities are still ongoing, hence the current evaluation serves more as a mid-term review of project progress with the goal of identifying potential areas for improvement.
Evaluation Question 2. Overall, OES-funded Lacey Act Amendment activities are appropriate to the host country needs and conditions, with variations in the extent of fit across countries. However, OES should pay closer attention to host country conditions and external factors, for example, the lack of resources among host countries and corruption which affect results.
Evaluation Question 3. The XyloTron and its scientific image reference collection is the most successful of OES’ current activities and offers huge potential to tackle the shipment and trading of illegal timber. Its success can be partially attributed to the fit of the technology to host country conditions and needs. On the contrary, implementing partners cited the withdrawal of activities in Far East Russia when they had been yielding results as most disappointing.
Evaluation Question 4. There is a certain level of coordination between OES’ implementing partners, especially between DOJ and USFS in coordinating logistics and trainers at DOJ’s workshops. OES also co-funds USFS’ activities with USAID and TNC’s activities with the Australian government. Beyond that, most activities happen in parallel and even overlap to a certain degree. An example is between DOJ and other agencies conducting training of law enforcement agencies, e.g., INL and Interpol.
Evaluation Question 5. Interviewees spoke highly of the XyloTron’s early results and huge potential. Many interviewees across agencies and levels recommended the continuation and even expansion of XyloTron’s funding. However, findings suggest the need to streamline OES’ funding mechanism to reduce administrative oversight burden.
Evaluation Question 6. NGO representatives suggested that OES shift funding to support longer-term projects. Ideally, projects should work with the same beneficiaries consistently, over longer periods of time, and in the same country to ensure that projects are achieving their outcomes. Multiple stakeholders also pointed to the need for OES to more strategically select host countries based on need and appropriateness of country context and intervention activities.
Recommendations
The following are key recommendations based on findings and conclusions of the data, to help OES and the implementers (DOJ, TNC, and USFS) improve the effectiveness and efficiency of programming in the near future.
Target Countries for Implementation based on Strategy and Need
1. Strategically select host countries. Consider adopting a more strategic approach to the selection and targeting of host countries to implement Lacey Act Amendment activities, including selecting countries based on the appropriateness of the activity to the country context and targeting major suppliers and exporters of illegal timber and producers of finished wood products.
2. Assess the possibility of re-engaging with Russia counterparts and expanding activities in China. OES should assess the potential to reinitiate activities in Far East Russia focused on preventing the continued illegal harvesting and trade of Mongolian oak and other endangered wood species, and to expand its activities in China. Where political or diplomatic sensitivities are unavoidable, OES should identify alternative ways to support activities in these two countries, for example, by partnering with agencies or organizations that have the capability to expand activities.
Adopt a Longer-Term and More Streamlined Approach to OES’ Funding Mechanism
3. Create an internal long-term strategic plan to guide OES’ funding streams. If not already developed, OES should create an internal five-year strategic plan that will guide the direction of OES funding.
4. Consider longer-term funding mechanisms. If possible, OES should consider administering multi-year IAAs and awards to implementing agencies to fulfill project objectives that align with OES’ internal strategic plan. At the same time, OES should take a more targeted funding approach by reducing the number of implementing agencies funded and the number of IAAs active at any one time, allowing OES to strategically support select interventions.
5. Streamline OES’ funding agreements to reduce administrative burden. OES should work to streamline funding agreements across USG implementing partners to reduce inefficiencies and administrative oversight burden.
6. Publish OES funding cycle and opportunities broadly. OES might consider posting funding opportunities on its website and www.grants.gov website, and also broadly promote funding opportunities to contacts and interest groups to reach a broader range of interested parties (both USG agencies and NGOs) and to elicit a wider range of ideas to address illegal logging.
7. Assess the role of TNC in OES’ funding portfolio. TNC stands out among OES’ current implementing agencies because of the nature and reach of its activities. OES funding for TNC’s RAFT program is small compared to funding from the Australian government but represents important collaboration among consumer countries to combat illegal logging. OES funding for RAFT also is important since TNC directly engages in China, an important country in the global issue of illegal logging.
Increase Potential Impact of Lacey Act Amendment Programming
8. Conduct a series of training with the same participants at the same location. To yield impact, DOJ should implement a series of training workshops, in partnership with a local/regional NGO to ensure sustainability, over one to two years targeting the same law enforcement counterparts in the same country.
9. Expand the pool of qualified trainers. DOJ should regularly conduct training-of-trainers to develop a roster of prosecutors and investigators from ENRD, FWS, Animal and Plant Health Inspection Service (APHIS), and the Customs and Border Patrol (CBP) who can effectively deliver training workshops for host country law enforcement counterparts. DOJ should consider providing training-of-trainers to local/regional NGOs to promote sustainability.
10. Design an online foundational training on the Lacey Act Amendment that could serve as an optional pre-training prior to the in-person training. DOJ should consider designing and implementing an online training course that could be used as a prerequisite for participation in the training workshops in countries with reliable internet access. It would be useful for foundational information to be provided in multiple languages, especially Spanish, Portuguese, Russian, Chinese, Bahasa Indonesian, and the languages of other key countries involved in the trade of illegal timber.
11. Identify ways to support domestic enforcement of the Lacey Act Amendment. OES should investigate improving enforcement of the Lacey Act Amendment through CBP, APHIS, and FWS, which conduct domestic inspections and investigations at U.S. entry ports and borders.
12. Continue and consider increasing funding for the XyloTron. Given its huge potential to impact compliance with and enforcement of the Lacey Act Amendment, OES should continue and even increase its funding to FPL for the refinement of the XyloTron and scientific image reference collection, ideally to the stage where it is ready for industry use.
13. Improve monitoring and reporting requirements of implementing agencies. In order to assess progress and results of OES-funded Lacey Act Amendment activities, OES should work with its partners to refine their monitoring and reporting requirements.
14. Consider bridge funding the FLA. Given Forest Legality Alliance’s (FLA) role as an information hub—bringing together different stakeholders from the public, private, and NGO sectors to share and collaborate—OES should identify ways to provide bridge or long-term funding to FLA.
15. Work with local community and civil society organizations. Given the level at which community and civil society organizations operate, illegal activities taking place in villages and communities are typically more visible to these organizations than to host country government agencies.
16. Meet with key NGO stakeholders on a regular basis. OES might consider holding regular stakeholder meetings with NGOs to raise awareness within NGOs of ongoing Lacey Act Amendment activities and elicit ideas for future activities.
Improve Coordination between Stakeholders
17. Encourage coordination between implementing and donor agencies. OES should consider bringing implementing and donor agencies together on a regular basis to coordinate activities, align and prioritize strategies, and resolve challenges. These meetings serve as space to highlight the unique strengths and weaknesses of different agencies and provide the opportunity to coordinate activities in a complementary way.
18. Learn from efforts to combat wildlife trafficking. While opinions diverge on whether efforts against illegal logging should be combined with initiatives to counter wildlife trafficking, there are lessons to be learned from existing efforts against wildlife trafficking. OES might sponsor a workshop with representatives from the WWF (World Wildlife Fund for Nature), CITES, WRI (World Resource Institute), and other relevant NGOs to discuss approaches that might be useful.
———-
[1] The Convention on International Trade in Endangered Species (CITES) is a global conservation agreement to which 181 counties are a party.
[2] One interview was conducted in December 2015.