Honey bees (Apis mellifera) have recently experienced higher than normal overwintering colony losses. Many factors have been evoked to explain the losses, among which are the presence of residues of pesticides and veterinary products in hives. Multiple residues are present at the same time, though most often in low concentrations so that no single product has yet been associated with losses. Involvement of a combination of residues to losses may however not be excluded. To understand the impact of an exposure to combined residues on honey bees, we propose a mechanism-based strategy, focusing here on Multi-Drug Resistance (MDR) transporters as mediators of those interactions.
Using whole-animal bioassays, we demonstrate through inhibition by verapamil that the widely used organophosphate and pyrethroid acaricides coumaphos and τ-fluvalinate, and three neonicotinoid insecticides: imidacloprid, acetamiprid and thiacloprid are substrates of one or more MDR transporters. Among the candidate inhibitors of honey bee MDR transporters is the in-hive antibiotic oxytetracycline. Bees prefed oxytetracycline were significantly sensitized to the acaricides coumaphos and τ-fluvalinate, suggesting that the antibiotic may interfere with the normal excretion or metabolism of these pesticides.
Many bee hives receive regular treatments of oxytetracycline and acaricides for prevention and treatment of disease and parasites. Our results suggest that seasonal co-application of these medicines to bee hives could increase the adverse effects of these and perhaps other pesticides. Our results also demonstrate the utility of a mechanism-based strategy. By identifying pesticides and apicultural medicines that are substrates and inhibitors of xenobiotic transporters we prioritize the testing of those chemical combinations most likely to result in adverse interactions.
Citation: Hawthorne DJ, Dively GP (2011) Killing Them with Kindness? In-Hive Medications May Inhibit Xenobiotic Efflux Transporters and Endanger Honey Bees. PLoS ONE 6(11): e26796. doi:10.1371/journal.pone.0026796
Editor: Guy Smagghe, Ghent University, Belgium
Received: December 3, 2010; Accepted: October 4, 2011; Published: November 2, 2011
Copyright: © 2011 Hawthorne, Dively. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by the United States Department of Agriculture. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Honey bees are in trouble. Widespread depopulation of colonies often characterized by high overwintering losses has occurred since at least 2006 in the United States, threatening the sustainability of North American apiculture. Despite considerable effort, no single cause of the phenomenon called colony collapse disorder (CCD) has been identified, though associations of several pathogens and parasites appear to increase the risk of colony collapse , . Pesticides are also among the suspected contributing factors of colony collapse both because bees encounter a diverse array of pesticides when foraging and because more than 120 different pesticides have been found within bee hives , , , , . Some pesticides have received extra scrutiny, notably the acaricides coumaphos and τ-fluvalinate, applied to bee hives for control of parasitic varroa mites, and the widely used neonicotinoid insecticides. These acaricides are applied directly to bee hives, accumulate in wax and were found in nearly all hives recently tested in both N. American and France , . The neonicotinoids (especially imidacloprid) are of concern because they are toxic to honey bees, used on many crops and ornamental plants, and they tend to be systemically distributed within treated plants, potentially contaminating nectar and pollen of treated and rotational crops not initially treated with these products , , , .
Although pesticide drift and overdosing cause accidental bee kills no single pesticide has been directly implicated with widespread overwintering losses or CCD , . It remains possible however, that combinations of toxins may cause adverse additive or synergistic effects that would be difficult to detect through surveys of beekeepers or analysis of their apiaries without dedicated multifactorial analysis. It has been shown, for example, that the toxicity to bees of some pyrethroid and neonicotinoid insecticides increases significantly when combined with certain fungicides , . Similarly, Johnson et al.  found that coumaphos and τ-fluvalinate each synergize the other's toxicity to honey bees, perhaps through competitive inhibition of the metabolic enzymes that detoxify those pesticides. Given the many pesticides that bees encounter there may be adverse combinations of them eroding hive health in both subtle and dramatic ways.
The problem, of course, is the large number of potentially adverse pesticide combinations which prevents evaluation of all, or even most, combinations of them. This problem challenges our ability to anticipate the risks associated with bee's exposure to a novel pesticide or to identify combinations of toxins contributing to a colony collapse. If we could identify mechanisms of the honey bee xenobiotic metabolism and excretion systems that systematically mediate multiple-toxin interactions, we could reduce the overwhelming number of candidate pesticide interactions to a smaller set of compounds that are substrates or inhibitors of the most predictive mechanisms.
The membrane-bound transporter proteins from the ABC transporter family of proteins are found in all phyla , . The xenobiotic transporters in this family actively shuttle toxins across cell membranes to reduce the intracellular toxin and metabolite concentrations. Working in concert with metabolic enzymes, these transporters mediate a baseline tolerance to a diverse array of toxins including numerous drugs, pesticides and phytochemicals , . Several of these transporters, especially members of the ABCB, ABCC, and ABCG subfamilies of transporters (referred to here as Multiple Drug Resistance, or MDR transporters), are of medical importance, playing a role in resistance to multiple cancer and anti-parasite drugs , , .
MDR transporters are relatively unstudied in insects, and completely neglected in honey bee toxicology. These transporters act in several insect tissues, including the cuticle , malpighian tubules , , midgut  and at the blood-brain barrier ,  to transport toxins, including pesticides, towards excretion . The honey bee genome contains genes coding for orthologues of these proteins, which presumably protect bees from toxins as they do in Drosophila melanogaster , , , chironomid flies , mosquitoes , Heliothis virescens (tobacco budworm)  and Manduca sexta (tomato hornworm) , . It seems reasonable therefore to consider the role that these proteins play in honey bee tolerance of pesticides and to begin an analysis of potentially inhibitory compounds that bees commonly encounter.
The most well studied MDR transporter, p-glycoprotein (p-gp), has both a diverse range of substrates and is inhibited by an array of drugs, pesticides and plant compounds . This inhibition is a mechanism by which MDR transporters would cause adverse interactions among many chemicals; one compound inhibits the transporters thereby increasing sensitivity to other toxic substrates. The drug verapamil is a potent inhibitor of p-gp and possibly other MDR transporters , . It is frequently used as the standard inhibitor of p-gp where it increases the sensitivity of treated cells, tissues or organisms to toxic transporter substrates , , . Here we use verapamil inhibition to determine if 5 pesticides are substrates of MDR transporters and therefore potentially synergized by other inhibitors more likely to be encountered by honey bees. Remarkably, three widely used in-hive pesticides and medications (the previously mentioned acaricides coumaphos and τ-fluvalinate and the antibiotic oxytetracycline) are known substrates and/or inhibitors of mammalian p-gp , , . We suspect that these in-hive medications and pesticides may be interacting with bee's MDR transporters, increasing their sensitivity to these and perhaps other pesticides and toxins. The frequent contamination of hive wax with these acaricides  and routine treatment of hives with oxytetracycline , , ,  undoubtedly increases the exposure of bees to these compounds, with potentially significant consequences if they are indeed substrates or inhibitors of honey bee MDR transporters.
Interaction of neonicotinoid insecticides with insect MDR transporters has not yet been reported. Because of the likelihood of exposure of bees to these insecticides we ask if the neonicotinoid insecticides imidacloprid, acetamiprid and thiacloprid are substrates of honey bee MDR transporters. Evidence of neonicotinoid processing by MDR transporters would be significant because inhibition of those transporters could cause mortality at lower doses than normally expected for individual compounds.
When fed to bees verapamil significantly increased the toxicity of all 5 acaricides/insecticides. Mean mortality of young worker bees topically treated with the acaricides coumaphos or τ-fluvalinate was significantly higher when bees were pretreated with verapamil (Fig. 1, Table 1). Control mortality following topical application of acetone was 0% for both sucrose and sucrose+verapamil fed bees. Acute oral toxicity was also significantly higher for all three neonicotinoids (acetamiprid, thiacloprid, imidacloprid) when bees were pretreated with verapamil (Fig. 1, Table 2). Increased mortality at higher concentrations and at the later end point (48 h) was observed for thiacloprid, and at 48 h for imidacloprid. The effect of verapamil pretreatment did not differ among concentrations of these insecticides (Table 2). Control mortality of sucrose only and sucrose+verapamil cohorts averaged 2–3%.
Figure 1. Verapamil synergizes honey bee mortality by five acaricides/insecticides.
Mean mortality (±SE) of honey bees (average of 24 and 48 h) following topical (A, B) and oral (C, D, E) exposure to pesticides. Bees were pre-fed sucrose or sucrose+verapamil (1 mM) solution. For each pesticide, different letters indicate significant differences between treatments (p<0.05).doi:10.1371/journal.pone.0026796.g001
Table 1. Repeated-measures analysis of variance of honey bee mortality.doi:10.1371/journal.pone.0026796.t001
Table 2. Repeated-measures analysis of variance of honey bee mortality.doi:10.1371/journal.pone.0026796.t002
Oxytetracycline significantly increased the mortality of bees exposed to coumaphos and τ-fluvalinate (Fig. 2). For comparison with the verapamil synergism reported above, mean mortality of bees treated with 2 ug/ul coumaphos increased from 7% (n = 4 cages) to 51% (n = 4 cages) following feeding of OTC (1.4 mM), a significant but smaller increase than that caused by verapamil (Fig. 2A,Table 1). OTC feeding increased the mortality of bees treated with 3 ug/ul τ-fluvalinate from 5.6% (n = 10 cages) to 39% (n = 8 cages) (Fig. 2B, p = 0.002). Mean mortality of cohorts fed OTC alone were below 10% and were not significantly different from those fed sucrose alone (Fig. 2).
Figure 2. Oxytetracycline (OTC) synergizes honey bee mortality by in-hive acaricides.
Mean mortality (±SE) of honey bees pre-fed sucrose solution (50%) or sucrose+oxytetracycline (1.4 mM) and topical application of (A) coumaphos (average of 24 and 48 h) and (B) τ-fluvalinate (24 h). For each pesticide, different letters indicate significant differences between treatments (p<0.05).doi:10.1371/journal.pone.0026796.g002
Here we provide the first evidence that the MDR transporter(s) inhibited by verapamil play a role in protecting honey bees from pesticides, and that the acaricides coumaphos and τ-fluvalinate, and 3 neonicotinoid insecticides are substrates of these transporters in insects. The observation that coumaphos and τ-fluvalinate are substrates of honey bee p-gp or another MDR transporter was anticipated from previous study of mouse cells, and suggests that insect and mammalian MDR transporters share substrates. Clearly, the abundance of these pesticides found in the wax and pollen of bee hives  coupled with evidence that their toxicity to bees is increased through inhibition of MDR transporters implicates them as toxins of interest in any multifactorial explanation of high overwintering colony losses.
This is the first report that neonicotinoid insecticides are substrates of insect MDR transporters. In efforts to protect honey bees, energetic opposition to the neonicotinoids has arisen in North America and Europe, but direct implication of them in overwintering losses has not been sustained by recent research , . Estimates of the environmental exposure of bees to imidacloprid are typically low relative to the LD50 , and studies have not demonstrated hive-level consequences of imidacloprid contamination . Our results suggest that inhibition of MDR transporters may reduce the LD50 of neonicotinoids possibly amplifying acute and chronic effects to bees at lower concentrations.
The large increases in sensitivity to pesticides by inhibition of MDR transporters and the chemical diversity of the synergized pesticides suggest that these transporters may mediate adverse synergisms of diverse toxins in bees. Because of its clinical importance in human health, proven and candidate p-gp substrates and inhibitors of many types have been identified , . Knowledge of these compounds may help us identify chemicals likely to interact with honey bee MDR transporters. In the first application of this mechanism-based strategy to honey bees, we uncover a significant negative interaction among three medications routinely applied to bee hives , , . OTC, coumaphos and τ-fluvalinate are all known to interact with mammalian p-gp , , . If honey bee transporters behave similarly, we would expect increased toxicity following co-application of a toxic transporter substrate and an inhibitor. As anticipated, concentrations of OTC similar to those applied to bee hives increased bee's sensitivity to both coumaphos and τ-fluvalinate. OTC is applied to bee hives in the late fall and/or early spring, often in tandem with one of the acaricides . Our results suggest that co-application of these compounds could increase the likelihood of intoxication by the acaricides and other pesticides contaminating beeswax and food stores. These results raise the possibility that adverse interactions of medications (such as OTC) and pesticides (coumaphos and τ-fluvalinate) contribute to the loss of honey bee colonies during the winter or early spring, a common feature of CCD. Although the per-bee concentration of OTC used here is similar to field application rates, the pesticide concentrations are higher than those found in bee hives (see ). Therefore, although we have demonstrated that verapamil and OTC increase bee's sensitivity to these pesticides in acute laboratory bioassays, additional testing of lower pesticide concentrations over longer time periods is necessary to fully understand the field relevance of these interactions. Additional work is also required to directly demonstrate that OTC inhibits p-gp or other efflux transporters in honey bees. Nevertheless, we show here using OTC and the acaricides as an example, how identification of MDR transporter substrates and inhibitors can highlight potentially dangerous chemical combinations and improve the assessment and management of toxicological risks faced by honey bees.
Materials and Methods
Bees were collected for laboratory bioassays from newly established colonies reared on new frames and freshly drawn comb. Colonies were not treated with apicultural medications or pesticides. Frames with emerging workers were taken from hives and placed into dark growth chambers maintained at 33±2°C and (70–80%) RH. Newly-emerged bees were collected from the frames daily and maintained in groups of 20–30 in 80×100 mm metal mesh cages capped at each end by standard polystyrene petri dishes. Bees were fed sucrose solution (50%; w:v) through 1 mm holes from a 2.0 ml microfuge tube.
Terramycin (oxytetracycline, 5.5% soluble powder, Pfizer) was purchased from Dadant and Sons (Hamilton, Illinois). Coumaphos, τ-fluvalinate (both technical grade) and verapamil were purchased from Sigma-Aldrich Inc. (St. Louis, MO). Commercial formulations of imidacloprid (Admire Pro) and thiacloprid (Calypso) were provided by Bayer CropScience (Durham, NC), and acetamiprid (Assail) was provided by United Phosphorous Inc. (King of Prussia, PA).
Verapamil (1 mM) and oxytetracycline (OTC, 1.4 mM) were incorporated into 50% sucrose solutions for oral dosing of 1–3 day old workers. Preliminary feeding trials of 1 mM solutions of oral verapamil revealed no toxicity. The 1.4 mM concentration of OTC provides a per-bee exposure comparable to that of the label-recommended dosage of 600 mg applied to a hive containing 12,000 bees—a typical colony size entering winter . Sucrose-drug solutions were made fresh every 3 days and the vials supplying each cage were replaced as needed.
Topical bioassays of insecticides/acaricides
Cohorts of 3–6 day old workers pretreated by feeding with the two sucrose-drug solutions were anesthetized with CO2, and 1 ul of coumaphos (2 ug/ul)or τ-fluvalinate (3 ug/ul) in acetone (or acetone alone for controls) was applied to the dorsal thorax of each bee using an ISCO microapplicator driving a 1/4 cc tuberculin syringe. After application, bees were returned to cages containing the sucrose-drug or sucrose-only solution. Mortality of bees in each cage was recorded at 24 and 48 h. 5–10 replicate cohorts of 25 bees were tested for each acaricide - pretreatment combination.
Oral bioassays of insecticides
Pre-fed cohorts of 3–6 day old workers were fed sucrose syrup containing one of the neonicotinoids. Mortality of each cage was recorded at 24 and 48 hours. Imidacloprid was tested at 5 and 50 ng/ul, acetamiprid at 25 and 100 ng/ul, and thiacloprid at 25, 100 and 500 ng/ul. These concentrations caused low-intermediate mortality of bees fed sucrose-only solution in preliminary range-finding experiments. 2–13 replicate cohorts of 25 bees were tested for each toxin concentration - pretreatment combination.
The effects of verapamil or OTC pretreatment on insecticide/acaricide mortality were tested using a repeated measures analysis of variance (Proc Mixed, SAS). Following transformation (arcsine square-root), mortality was analyzed with a model that included pretreatment, insecticide concentration if multiple levels were used, and time endpoints (24 and 48 h) as fixed factors to assess the main effects and their interactions. Because only mortality at 24 h was available, analysis of τ-fluvalinate combined with OTC, was performed using a simple t-test, comparing the τ-fluvalinate and the τ-fluvalinate+OTC treatments.
We thank Jeffrey Pettis, Mike Embrey, Peter Swaan, Jeff Scott and David Onstad for thoughtful discussions and suggestions.
Conceived and designed the experiments: DJH. Performed the experiments: DJH. Analyzed the data: DJH GPD. Contributed reagents/materials/analysis tools: DJH GPD. Wrote the paper: DJH.
- 1. Cox-Foster DL, Conlan S, Holmes EC, Palacios G, Evans JD, et al. (2007) A metagenomic survey of microbes in honey bee colony collapse disorder. Science 318: 283–287.
- 2. vanEngelsdorp D, Evans JD, Saegerman C, Mullin C, Haubruge E, et al. (2009) Colony collapse disorder: a descriptive study. PLoS ONE 4: e6481.
- 3. Ratnieks FLW, Carreck NL (2010) Clarity on honey bee collapse? Science 327: 152–153.
- 4. Frazier M, Mullin C, Frazier J, Ashcraft S (2008) What have pesticides got to do with it? Am Bee J 148: 521–523.
- 5. Chauzat MP, Carpentier P, Martel AC, Bougeard S, Cougoule N, et al. (2009) Influence of pesticide residues on honey bee (Hymenoptera: Apidae) colony health in France. Environ Entomol 38: 514–523.
- 6. Mullin CA, Frazier M, Frazier JL, Ashcraft S, Simonds R, et al. (2010) High levels of miticides and agrochemicals in North American apiaries: implications for honey bee health. PLoS ONE 5: e9754.
- 7. Maus C, Cure G, Schmuck R (2003) Safety of imidacloprid seed dressings to honey bees: a comprehensive overview and compilation of the current state of knowledge. Bull Insectol 56: 51–57.
- 8. Rortais A, Arnold G, Halm MP, Touffet-Briens F (2005) Modes of honeybees exposure to systemic insecticides: estimated amounts of contaminated pollen and nectar consumed by different categories of bees. Apidologie 36: 71–83.
- 9. Laurent FM, Rathahao E (2003) Distribution of [14C]imidacloprid in sunflowers (Helianthus annuus L.) following seed treatment. J Agric Food Chem 51: 8005–8010.
- 10. Elbert A, Haas M, Springer B, Thielert W, Nauen R (2008) Applied aspects of neonicotinoid uses in crop protection. Pest Manage Sci 64: 1099–1105.
- 11. Iwasa T, Motoyama N, Ambrose JT, Roe RM (2004) Mechanism for the differential toxicity of neonicotinoid insecticides in the honey bee, Apis mellifera. Crop Protect 23: 371–378.
- 12. Pilling ED, Jepson PC (1993) Synergism between EBI fungicides and a pyrethroid insecticide in the honeybee (Apis mellifera). Pestic Sci 39: 293–297.
- 13. Johnson RM, Pollock HS, Berenbaum MR (2009) Synergistic interactions between in-hive miticides in Apis mellifera. J Econ Entomol 102: 474–479.
- 14. Gottesman MM, Fojo T, Bates SE (2002) Multidrug resistance in cancer: role of ATP-dependent transporters. Nat Rev Cancer 2: 48–58.
- 15. Sturm A, Cunningham P, Dean M (2009) The ABC transporter gene family of Daphnia pulex. BMC Genomics 10:
- 16. Didziapetris R, Japertas P, Avdeef A, Petrauskas A (2003) Classification analysis of P-glycoprotein substrate specificity. J Drug Targeting 11: 391–406.
- 17. Buss DS, Callaghan A (2008) Interaction of pesticides with p-glycoprotein and other ABC proteins: A survey of the possible importance to insecticide, herbicide and fungicide resistance. Pestic Biochem Physiol 90: 141–153.
- 18. Leslie EM, Deeley RG, Cole SPC (2005) Multidrug resistance proteins: role of P-glycoprotein, MRP1, MRP2, and BCRP (ABCG2) in tissue defense. Toxicol Appl Pharmacol 204: 216–237.
- 19. Kerboeuf D, Guegnard F (2011) Anthelmintics are substrates and activators of nematode p-glycoprotein. Antimicrob Agents Chemother 55: 2224–2232.
- 20. Lanning CL, Ayad HM, AbouDonia MB (1996) P-glycoprotein involvement in cuticular penetration of C-14 thiodicarb in resistant tobacco budworms. Toxicol Lett 85: 127–133.
- 21. Gaertner LS, Murray CL, Morris CE (1998) Transepithelial transport of nicotine and vinblastine in isolated malpighian tubules of the tobacco hornworm (Manduca sexta) suggests a P-glycoprotein-like mechanism. J Exp Biol 201: 2637–2645.
- 22. Leader JP, O'Donnell MJ (2005) Transepithelial transport of fluorescent p-glycoprotein and MRP2 substrates by insect Malpighian tubules: confocal microscopic analysis of secreted fluid droplets. J Exp Biol 208: 4363–4376.
- 23. Tapadia MG, Lakhotia SC (2005) Expression of mdr49 and mdr65 multidrug resistance genes in larval tissues of Drosophila melanogaster under normal and stress conditions. Cell Stress Chaperones 10: 7–11.
- 24. Mayer F, Mayer N, Chinn L, Pinsonneault RL, Kroetz D, et al. (2009) Evolutionary conservation of vertebrate blood-brain barrier chemoprotective mechanisms in drosophila. J Neurosci 29: 3538–3550.
- 25. Murray CL, Quaglia M, Arnason JT, Morris CE (1994) A putative nicotine pump at the metabolic blood-brain-barrier of the tobacco hornworm. J Neurobiol 25: 23–34.
- 26. Callaghan A, Denny N (2002) Evidence for an interaction between p-glycoprotein and cadmium toxicity in cadmium-resistant and -susceptible strains of Drosophila melanogaster. Ecotoxicol Environ Saf 52: 211–213.
- 27. Vache C, Camares O, Cardoso-Ferreira MC, Dastugue B, Creveaux I, et al. (2007) A potential genomic biomarker for the detection of polycyclic aromatic hydrocarbon pollutants: Multidrug resistance gene 49 in Drosophila melanogaster. Environ Toxicol Chem 26: 1418–1424.
- 28. Podsiadlowski L, Matha V, Vilcinskas A (1998) Detection of a P-glycoprotein related pump in Chironomus larvae and its inhibition by verapamil and cyclosporin A. Comparative Biochemistry and Physiology B-Biochemistry & Molecular Biology 121: 443–450.
- 29. Porretta D, Gargani M, Bellini R, Medici A, Punelli F, et al. (2008) Defence mechanisms against insecticides temephos and diflubenzuron in the mosquito Aedes caspius: the P-glycoprotein efflux pumps. Med Vet Entomol 22: 48–54.
- 30. Tsuruo T, Iida H, Tsukagoshi S, Sakurai Y (1981) Overcoming of vincristine resistance in P388 leukemia invivo and invitro through enhanced cyto-toxicity of vincristine and vinblastine by verapamil. Cancer Res 41: 1967–1972.
- 31. Bain LJ, McLachlan JB, LeBlanc GA (1997) Structure-activity relationships for xenobiotic transport substrates and inhibitory ligands of P-glycoprotein. Environ Health Perspect 105: 812–818.
- 32. Bain LJ, LeBlanc GA (1996) Interaction of structurally diverse pesticides with the human MDR1 gene product P-glycoprotein. Toxicol Appl Pharmacol 141: 288–298.
- 33. Schrickx J, Fink-Gremmels J (2007) P-glycoprotein-mediated transport of oxytetracycline in the Caco-2 cell model. J Vet Pharmacol Ther 30: 25–31.
- 34. Spivak M, Reuter GS (2001) Resistance to American foulbrood disease by honeybee colonies Apis mellifera bred for hygienic behavior. Apidologie 32: 555–565.
- 35. Sammataro D (1998) The beekeeper's handbook. Ithaca, NY: Comstock Pub. 190 p.
- 36. Sanford MT, Bonney RE (2010) Storey's Guide to Keeping Honey Bees. North Adams, MA: Storey Publishing.
- 37. Delaplane KS, Lozano LF (1994) Using Terramycin(R) in honey-bee colonies. Am Bee J 134: 259–261.
- 38. Faucon JP, Aurieres C, Drajnudel P, Mathieu L, Ribiere M, et al. (2005) Experimental study on the toxicity of imidacloprid given in syrup to honey bee (Apis mellifera) colonies. Pest Manage Sci 61: 111–125.
- 39. Winston ML (1987) The biology of the honey bee. Cambridge, Mass.: Harvard University Press. 281 p.