Intelligence and Security Informatics Data Sets

AZSecure-data.org

Data Infrastructure Building Blocks for ISI. A Project of the University of Arizona (NSF #ACI-1443019), Drexel University,

University of Virginia, University of Texas at Dallas, and University of Utah

GeoWeb Forums

 

​The sixty-five forums available here are presented by country. Each forum contains postings organized into threads that generally indicate the topic under discussion. Each posting includes detailed metadata such as date, member name, etc. The forums were collected up through 2010-2012 by the Artificial Intelligence Lab to support its Geopolitical Web ("GeoWeb") Research project. GeoWeb investigated computational approaches for monitoring public opinion in regions of conflict, assessing country risk indicators in the social media of fragile or weakening states, and correlating these risk signals with commonly accepted quantitative geopolitical risk assessments. Collection sizes vary, with smaller forums containing a few thousand or a few tens of thousands of postings, and larger collections boasting hundreds of thousands to over a million postings. Languages include English, Arabic, French, Indonesian, Pashto, or Urdu, depending on the forum. Information regarding the focus of each forum is not recorded; they were collected for being of general interest, with discussions covering a wide range of topics.


The list below provides additional information about each forum as well as links to the forum files. Each forum is provided as a downloadable compressed text file which may then be opened in any CSV-compatible text processing program. To download, click on the name of the forum and choose to open or save the file when prompted by your browser. Sizes given are approximate.  


AFGHANISTAN

  • AfghanForum - Language: English. Dates:10/2/2007 - 5/2/2011; Posts: 88,005; Threads: 4,826; Members: 1,114. Zip file: 12.9 MB zipped, 53.3 MB unzipped.

  • Afghanforums - Language: English. Dates: 04/08/2004 - 05/06/2011; Posts: 151,560; Threads:4,598; Members: 980. Zip file: 18 MB zipped, 57 MB unzipped.

  • ​PashtunForums - Language: English. Dates: 3/5/2010 - 5/4/2011; Posts: 72,750; Threads: 3,023; Members: 481. Zip file: 10 MB zipped, 19 MB unzipped.

  • Tolafghan - Language: Pashto. Dates: 10/13/2002 - 04/09/2011; Posts: 32,257; Threads: 4,709; Members: 1,592. Zip file: 21 MB zipped, 83 MB unzipped.


ALGERIA

  • Eshamel - Language: Arabic. Dates: 7/6/2005 - 11/14/2011. Posts: 103,806; Threads: 28,754; Members: 3,514. Zip file: 41.72 MB zipped, 156 MB unzipped.

  • Etudiantdz - Language: Arabic. Dates: 11/28/2007 - 09/09/2011. Posts: 177,652; Threads: 31,384; Members: 12,148. Zip file: 130 MB zipped, 541 MB unzipped.

  • ​Ngaous - Language: Arabic. Dates: 4/22/2009 - 3/29/2012; Posts: 147,891; Threads: 77,605; Members: 1,899. Zip file: 51 MB zipped, 212 MB unzipped.


EGYPT

  • Alnilin - Language: Arabic. Dates: 8/2/2003 - 10/31/2011; Posts: 8,901; Threads: 817; Members: 135. Zip file: 2.63 MB zipped, 11.7 MB unzipped.

  • Damanhour - Language: Arabic. Dates: 5/24/2009 - 12/7/2011; Posts: 23,142; Threads: 9,559; Members: 1,637. Zip file: 10.5 MB zipped, 40.4 MB unzipped.

  • Egyppl - Language: Arabic. Dates: 10/18/2007 - 11/21/2011; Posts: 321,015; Threads: 61,269; Members: 36,264. Zip file: 76.69 MB zipped, 311 MB unzipped.

  • EgyptianTalks - Language: Arabic. Dates: 12/29/2000 - 09/09/2011; Posts: 644,290; Threads: 51,869; Members: 4,724. Zip file: 226 MB zipped, 833 MB unzipped.

  • Forsanelhaq - Language: Arabic. Dates: 10/01/2005 - 09/09/2011; Posts: 131,108; Threads: 20,648; Members: 2,875. Zip file: 378 MB zipped, 1.63 GB unzipped.


INDONESIA

  • ForumBebas - Language: Indonesian. Dates: 11/10/2007 - 09/09/2011; Posts: 7,165; Threads: 278; Members: 1,627. Zip file: 544 KB zipped, 2.1 MB unzipped.

  • Kafegaul - Language: Indonesian. Dates: 2/18/2000 - 1/31/2011; Posts: 611,591; Threads: 31,001; Members: 5,021. Zip file: 13.31 MB zipped, 47.5 MB unzipped.

  • ​Kaskus - Language: Indonesian. Dates: 5/21/2008 - 1/23/2011; Posts: 419,977; Threads: 14,894; Members: 41,914. Zip file: 49 MB zipped, 216 MB unzipped.

  • Kompas - Language: Indonesian. Dates: 01/11/2008 - 04/05/2012; Posts: 184,914; Threads: 33,613; Members: 10,428. Zip file: 42 MB zipped, 133 MB unzipped.


IRAQ

  • AlForat - Language: Arabic. Dates: 12/14/2005 - 09/009/2010; Posts: 589,490; Threads: 100,553; Members: 36,914. Zip file: 65 MB zipped, 315 MB unzipped.

  • Dijlh - Language: Arabic. Dates: 10/01/2006 - 04/09/2012; Posts: 859,477; Threads: 123,545; Members: 21,249. Zip file: 161 MB zipped, 587 MB unzipped.

  • IragCenter - Language: Arabic. Dates: 9/29/2002 - 11/19/2011; Posts: 206,290; Threads: 61,334; Members: 4,092. Zip file: 160 MB zipped, 634 MB unzipped.

  • Iragiyat - Language: Arabic. Dates: 12/13/2007 - 1/18/2011 ; Posts: 280,380; Threads: 29,775; Members: 1,809. Zip file: 33.35 MB zipped, 147 MB unzipped.

  • IraqLights - Language: Arabic. Dates: 8/1/2005 - 12/28/2010; Posts: 303,419; Threads: 56,162; Members: 5,797. Zip file: 85.3 MB zipped, 335 MB unzipped.

  • IraqPF - Language: Arabic. Dates:7/27/2007 - 12/1/2010; Posts: 420,748; Threads: 25,348; Members: 7,136. Zip file: 26.08  MB zipped, 141 MB unzipped.

  • ​Rumaitha - Language: Arabic. Dates: 8/20/2007 - 1/18/2011; Posts: 54,163; Threads: 6,799; Members: 664. Zip file: 6 MB zipped, 31 MB unzipped.

  • ​Yahosein - Language: Arabic. Dates: 3/22/2002 - 2/22/2009; Posts: 214,616; Threads: 135,635; Members: 11,122. Zip file: 189 MB zipped, 726 MB unzipped.


JORDAN

  • Ana-Jordan- Language: Arabic. Dates: 6/12/2008 - 12/6/2011; Posts: 136,638; Threads: 15,081; Members: 1,840. Zip file: 20.73 MB zipped, 88.8 MB unzipped.

  • Qalbjo - Language: Arabic. Dates: 10/10/2007 - 04/09/2012; Posts: 381,574; Threads: 40,957; Members: 1,429. Zip file: 48 MB zipped, 188 MB unzipped.


LEBANON

  • Aljond - Language: Arabic. Dates: 4/12/2007 - 12/8/2011; Posts: 26,538; Threads: 101,542; Members: 37. Zip file: 35.11 MB zipped, 173 MB unzipped.

  • Ashefaa - Language: Arabic. Dates: 10/10/2005 - 04/09/2012; Posts: 609,030; Threads: 91,988; Members: 22,939. Zip file: 169 MB zipped, 706 MB unzipped.

  • Eljnoub - Language: Arabic. Dates: 10/10/2006 - 09/09/2011; Posts: 678,223; Threads: 100,181; Members: 4,505. Zip file: 129 MB zipped, 752 MB unzipped.

  • ​Lebanon - Language: Arabic. Dates: 6/11/2007 - 12/13/2011; Posts: 67,193; Threads: 19,164; Members: 1,231. Zip File: 17 MB zipped, 65 MB unzipped.


MOROCCO

  • Anaqamaghribia - Language: Arabic. Dates: 10/01/2006 - 09/09/2010; Posts: 255,667; Threads: 38,716; Members: 13,400. Zip file: 56 MB zipped, 215 MB unzipped.

  • Casafree - Language: French. Dates: 10/01/2004 - 09/09/2011; Posts: 32,302; Threads: 2,142; Members: 1,895. Zip file: 6.6 MB zipped, 21.5 MB unzipped.

  • GSM4Maroc - Language: Arabic. Dates: 11/17/2005 - 1/13/2011; Posts: 422,608; Threads: 38,563; Members: 12,257. Zip file: 81.75 MB zipped, 328 MB unzipped.

  • ​Rehabmaroc - Language: Arabic. Dates: 9/5/2007 - 1/13/2011; Posts: 148,872; Threads: 22,571; Members: 2,905. Zip file: 20 MB zipped, 87 MB unzipped.


PAKISTAN

  • Itdarasgah - Language: English. Dates: 10/01/2009 - 09/09/2011 Posts: 833,683; Threads: 66,858; Members: 6,504. Zip file: 60 MB zipped, 238 MB unzipped.

  • Itdunya - Language: English/Arabic. Dates: 11/9/2005 - 11/28/2011; Posts: 2,241,403; Threads: 186,844; Members: 29,258. Zip file: 190 MB zipped, 676 MB unzipped.

  • Iturdu - Language: English/Urdu. Dates: 04/21/2003 - 09/09/2011; Posts: 221,343; Threads: 10,533; Members: 759. Zip file: 11 MB zipped, 42 MB unzipped.

  • ​Pegham - Language: Urdu. Dates: 4/21/2005 - 11/15/2011; Posts: 526,739; Threads: 50,030; Members: 1,270. Zip file: 10 MB zipped, 30.8 MB unzipped.

  • UrduPages - Language: Urdu. Dates: 10/01/2003 - 09/09/2011; Posts: 927,562; Threads: 47,232; Members: 9,818. Zip file: 76 MB zipped, 254 MB unzipped.

  • ​UrduStuff - Language: Urdu. Dates: 6/18/2011 - 4/19/2012; Posts: 3,144; Threads: 1,061; Members: 59. Zip file: 535 KB zipped, 1.86 MB unzipped.


SAUDI ARABIA

  • Althuibi - Language: Arabic. Dates: 10/17/2008 - 12/7/2011; Posts: 132,476; Threads: 15,807; Members: 728. Zip file: 19.58 MB zipped, 85.4 MB unzipped.

  • Buraydahcity - Language: Arabic. Dates: 10/01/2001 - 09/09/2011; Posts: 2,686,247; Threads: 196,459; Members: 39,291. Zip file: 472 MB zipped, 1.73 GB unzipped.

  • ​Stop55 - Language: Arabic. Dates: 8/11/2003 - 12/16/2011; Posts: 1,380,592; Threads: 276,926; Members: 25,815. Zip file: 294 MB zipped, 1.2 GB unzipped.

  • www_7be - Language: Arabic. Dates: 10/01/2004 - 09/09/2011; Posts: 2,508,713; Threads: 132,988; Members: 82,298. Zip file: 304 MB zipped, 1.16 GB unzipped.


SOMALIA

  • AllSomaliForum - Language: English. Dates: 2/17/2007 - 8/1/2010; Posts: 53,945; Threads: 2,622; Members: 396. Zip file: 5.15 MB zipped, 19.5 MB unzipped.

  • Alsomal - Language: Arabic. Dates: 12/16/2005 - 7/29/2010; Posts: 209,321; Threads: 18,705; Members: 2,020. Zip file: 60 MB zipped, 239 MB unzipped.

  • Banadir - Language: English. Dates: 12/15/2008 - 8/3/2010 ; Posts: 427; Threads: 345; Members: 32. Zip file: .3 MB zipped, .8 MB unzipped.

  • Karbash - Language: English. Dates: 11/21/2005 - 08/09/2010; Posts: 134,973; Threads: 10,856; Members: 3,690. Zip file: 30 MB zipped, 116 MB unzipped.

  • SomaliaOnline - Language: English. Dates: 10/01/2001 - 02/09/2011; Posts: 282,590; Threads: 39,203; Members: 2,845. Zip file: 79 MB zipped, 218 MB unzipped.

  • Somalinet - Language: English. Dates: 6/22/2005 - 9/21/2010; Posts: 51,132; Threads: 8,933; Members: 1,482. Zip file:  7 MB zipped, 24 MB unzipped.

  • ​​SomaliUK - Language: English. Dates: 1/22/2001 - 7/24/2010; Posts: 56,339; Threads: 5,741; Members: 3,359. Zip file: 14 MB zipped, 42 MB unzipped.


TUNISIA

  • Marhba - Language: French. Dates: 01/21/2004 - 04/09/2012; Posts: 544,594; Threads: 26,039; Members: 11,973. Zip file: 79 MB zipped, 300 MB unzipped.

  • Matunisie - Language: French. Dates: 05/11/2000 - 04/09/2012; Posts: 57,432; Threads: 5,354; Members: 1,542. Zip file: 8.5 MB zipped, 30.6 MB unzipped.

  • ​Sama3y - Language: Arabic. Dates: 10/9/2005 - 4/17/2012; Posts: 90,902; Threads: 10,055. Zip file: 21 MB zipped, 90 MB unzipped.

  • ​​Shababtn - Language: Arabic. Dates: 1/1/2005 - 2/3/2012; Posts: 101,667; Threads: 7,423; Members: 23,993. Zip file: 11 MB zipped, 58 MB unzipped.


YEMEN

  • 3aden - Language: Arabic. Dates: 8/7/2004 - 7/4/2010; Posts: 16,115; Threads: 3,748; Members: 882. Zip file: 5 MB zipped, 20 MB unzipped.

  • Al-Reeem - Language: Arabic. Dates: 1/26/2009 - 7/20/2010; Posts: 103,748; Threads: 10,219; Members: 2,249. Zip file: 7.28 MB zipped, 43.7 MB unzipped.

  • Ayemen - Language: Arabic. Dates: 12/20/2005 - 7/19/2010; Posts: 126,055; Threads: 20,160; Members: 2,036. Zip file: 31 MB zipped, 134 MB unzipped.

  • Cyemen - Language: Arabic. Dates: 10/13/2002 - 8/14/2010; Posts: 82,400; Threads: 24,096; Members: 3,096. Zip file: 18.34 MB zipped, 70.1MB unzipped.

  • sa7t_ye - Language: Arabic. Dates: 10/01/2007 - 07/09/2010; Posts: 304,628; Threads: 20,938; Members: 1,479. Zip file: 45.3 MB zipped, 228 MB unzipped.

  • YemenM7BH (Yemen Forum)- Language: Arabic. Dates: 10/01/2006 - 07/09/2010; Posts: 455,025; Threads: 39,664; Members: 1,469. Zip file: 76 MB zipped, 356 MB unzipped.

  • YemenStudents - Language: Arabic. Dates: 10/01/2008 - 08/09/2010; Posts: 64,238; Threads: 18,803; Members: 713. Zip file: 16 MB zipped, 60.4MB unzipped.

  • ​Yemen7 - Language: Arabic. Dates: 11/22/2007 - 10/25/2010; Posts: 4,716; Threads: 788; Members: 145. Zip file: 1 MB zipped, 726 MB unzipped.

  • ​Yemen22 - Language: Arabic. Dates: 12/10/2006 - 7/17/2010; Posts: 78,377; Threads: 10,947; Members: 884. Zip file: 14 MB zipped, 62 MB unzipped.​

  • Yemen Gulf Cup 20 - Language: Arabic. Dates: 6/5/2009 - 7/25/2010; Posts: 8,205; Threads: 2,090; Members: 927. Zip file: 2 MB zipped, 8.5 MB unzipped.