What's new

DRDO hires Upscale's data mining for listening to online buzz

thestringshredder

FULL MEMBER
Joined
Jun 24, 2012
Messages
1,254
Reaction score
1
Country
India
Location
India
A lot is being written about the Defence Research and Development Organisation (DRDO) in the media these days. Almost every launch of AGNI missiles' test runs are published in local, national and international news platforms. Online blogs and news websites run a number of editorials about the DRDO. These shape minds of young India and create an image of Indian defence.
The DRDO has decided to disseminate information about the media coverage of Defence and S&T events related to it. A special effort to gather all possible media coverage during important events like missile launch, product release so as to track the response of the media on the events was initiated. The DRDO will probably use this dissection in shaping its public release policy. Also, this would be a good experience for the DRDO scientists who wish to remain updated.
The DRDO has hired services of Delhi-based technology consulting and data services firm “Upscale” which has its own data mining tool. This is a very unique technology. Unlike regular search engines, which crawl the entire web and present the search engine index in definite formats, Upscale's data miner crawl only specific to the need.
Based on Apache Solr, this tool crawls keyword specific articles only so that all those articles which follow a specific matrix of rules are accepted in the search engine index database. Once the news articles get profiled and crawled, they are presented to users with a rich UI experience such as chronological orders or topic/location wise.
A team of graduates from IIT Delhi, Delhi University, and Delhi Technological University have worked in development of this tool.
Upscale has decided to release this specific data mining tool under open source license so that further Research and Development on this can be carried out.
“Today's organizations accumulate huge volumes of data from a variety of sources on a daily basis. However, turning increasingly large amounts of data into useful insights and finding how to better utilize those insights in decision making remains a challenge for most,” says a text on SAS business website.
Data Mining is an interesting domain and more and more companies have started getting used to the idea of investing in it. The DRDO is just one of them.

Link - DRDO hires Upscale's data mining for listening to online buzz
 
. .
What exactly is the benefit of data mining?

exactly what it reads...you mine data.
You have huge amounts of data circulating on the social media sites...google search strings...facebook status messages...likes...photos,etc...
they all go into servers...companies use this data through Social business intelligence...they get customer insights and understand market realities straight from the horse's mouth (their existing or potential customers in this case talking about products movies services ideas etc)
I do not really understand what DRDO would want to achieve by this.
 
.
What exactly is the benefit of data mining?

Social network analysis by FBI -

Social network analysis (SNA) is often confused with social networking sites, such as Facebook, when in fact, SNA is an analytical tool that can be used to map and measure social relations. Through quantitative metrics and robust visual displays, police can use SNA to discover, analyze, and visualize the social networks of criminal suspects.


SNA, a social science methodology, serves as a valuable tool for law enforcement. While technologically sophisticated, SNA proves easy to employ. Using available data, police departments structure the examination of an offender’s social network in ways not previously possible.




Social network analysis provides a systematic approach for investigating large amounts of data on people and relationships.


Manual examination of social networks tends to be difficult, time consuming, and arbitrary, making it more prone to error. SNA provides a systematic approach for investigating large amounts of data on people and relationships. It improves law enforcement effectiveness and efficiency by using complex information regarding individuals socially related to suspects. This often leads to improved clearance rates for many crimes and development of better crime prevention strategies.



SNA derives its value from human organization and social interaction for criminal and noncriminal purposes. Social networks sometimes promote illegal behavior (e.g., juvenile delinquency and gang-related crime) among related offenders across criminal domains. They can provide a source for illicit drug and pornography distribution and international terrorism.1 The networks may supply an essential first condition for many serious criminal behaviors.



Social networks that enable crime are not mutually exclusive from the networks of law- abiding citizens. They are interspersed within these communities, drawing support from residents and extracting significant costs from host neighborhoods.2 The influence of social networks in producing criminal behavior indicates that effective crime-fighting strategies are contingent upon law enforcement’s ability to identify and respond appropriately to the networks where the behavior is embedded.



Theory and Method



SNA is a theory about how humans organize and a method to examine such organization. The approach indicates that actors are positioned in and influenced by a larger social network. Methodologically, it provides a precise, quantitative tool through which agencies can identify, map, and measure relationship patterns.



Three points of data—two actors and the tie or link between them—comprise the basic unit of analysis. Actors “nodes” are people, organizations, computers, or any other entity that processes or exchanges information or resources. Relationships “ties, connections, or edges” between nodes represent types of exchange, such as drug transactions between a seller and buyer, phone calls between two terrorists, or contacts between victims and offenders. SNA focuses on both positive and negative relationships between sets of individuals.



This analysis produces two forms of output, one visual and the other mathematical. The visual consists of a map or rendering of the network, called a social network diagram, which displays the nodes and relationships between them. In larger networks, key nodes are more difficult to identify; therefore, the analysis turns to the quantitative output of SNA.



The centrality of nodes, such as those representing offenders, identifies the prominence of persons to the overall functioning of the network. It indicates their importance to the criminal system, role, level of activity, control over the flow of information, and relationships. Basic centrality metrics provide further details. “Degree” gauges how many connections a particular node possesses, “betweenness” measures how important it is to the flow, and “closeness” indicates how quickly the node accesses information from the network. Nodes are rank ordered according to their centrality, with those at the top playing the most prominent role. These measures cannot tell an analyst what the structure should be, but they can elaborate on the actual makeup of the network. The value and actionable intelligence of each of these metrics is determined by the information the analyst wants.



Case Study



Social networks that enable crime are not mutually exclusive from the networks of law-abiding citizens.


In January 2008 a collaborative pilot project was launched to explore the viability of incorporating SNA into the precinct-level crime analysis methodologies of the Richmond, Virginia, Police Department (RPD). Participants included representatives of RPD, a university sociologist, and a software designer. The goal was for the research team, comprised of the sociologist and the software designer, to use crime data to assess how constructive SNA would be in solving the most prevalent crimes in the area and to determine the feasibility of training the precinct-level analysts to incorporate it into their workflow.



Researchers needed to determine what initiated violence between two groups of previously friendly young males. Several persons of interest, at one time on good terms, began to argue and assault one another. The source of the violence was not clear, and police were looking for ways to respond. They wanted to know if SNA could help them understand what sparked the violence and which strategies could be developed using a network approach.



The research team received access to RPD’s records management system to obtain information on criminal occurrences, arrests, criminal associates, demographics, and victim/offender relationships. The police provided no other background information on the individuals. The research team did not meet or discuss the ongoing investigation with the detectives. Analysis was done off-site, and the only recurring contact was with the police manager to extract the data in relational form.



Using 24 persons of interest labeled by a gang unit detective as “seeds”—starting points, or initially identified persons—the records management system extracted all connections among the seeds from 2007 through October 2008, proceeding four layers out and including any interconnections among the seed and nodes in each step. The connections were categorized by incident type—common incident participation, victim/offender, gang memberships, field contacts, involved others, common locations, and positive or negative connections.



Positive ties included a cooperative relationship between individuals, such as having family connections, robbing a store together, or hanging out. Negative ties indicated hostile relationships, such as those between a victim and offender. Individuals could have multiple and varying connections. Four networks resulted from the sampling, one for each layer out from the seeds. The networks included the seeds, the relationships among them, people directly connected to them, and those related to their associates. This involved 434 individuals and 1,711 ties. Several weak spots existed where a single node connected regions of the network and indicated dense areas of heavy interconnectivity.



Using SNA software, an analyst quickly produced a visual representation, including names, to assess the structure of the group or reference to whom a person of interest was connected. Through visual analysis and examination of the metric of betweenness, analysts located the source of the disagreement. The metric pointed to critical junctures in the network that revealed interpersonal tensions among males revolving around their relationships with females.



Two powerful male gang members reportedly had a positive relationship in October of 2007; however, in April 2008, one victimized a female friend of the other. During the same incident, this male also victimized the female friend of another male. Throughout the episode, a pattern emerged involving situations where a dominant male engaged with a female associate of another strong male. In other words, boys were fighting over girls.



Quantitative metrics provided additional information identifying the powerful players in this network. By rank ordering the individuals according to their centrality measures, the analysis confirmed that the gang unit was watching the right people and using community resources effectively. The metrics also helped unit members further analyze the importance of the seed nodes. Many of the nodes targeted by the unit ranked as powerful in the network based on an SNA metric. The quantitative metrics indicated six other vital players, including one critical to the flow of the network.



Results




Using social network analysis (SNA) software, an analyst can produce a visual representation to assess the structure of a group or reference to whom a person of interest is connected.

Unanticipated administrative processes delayed the timeline of the pilot project, making these results and recommendations too late to be actionable. The police already had solved the conflict. The knowledge of the detectives, which the research team was not privy to, validated the results. Officers confirmed that the answer the research team had discerned from the data—boys fighting over girls—was the cause of the conflict.



Detectives acknowledged that they would have solved the case more quickly and easily if they had this analysis available to guide their strategies. This feedback validated the worth of the approach and the usefulness of SNA and moved the project into the next phase. Precinct-level crime analysts received training in SNA through a 36-hour, in-house seminar. Through lectures and hands-on training, crime analysts from police and federal agencies used data from their own projects to learn to incorporate SNA to meet their needs. Within 2 weeks of completing the course, the analysts used SNA in several cases, including an aggravated assault/ shooting and several convenience store robberies.



In the shooting incident, the analyst used SNA to provide data on an associate of the suspect who previously was not noticed by the detective working the case. The analyst provided that information to the detective who used it to locate and interview that individual, which put additional pressure on the suspect, who was attempting to elude capture. This, combined with other social and financial pressures, caused the suspect to surrender.



Another case involved a string of convenience store robberies. Using an SNA map of a separate case, an analyst noticed a connection between a person of interest in the robberies in one precinct and a member of the network under investigation. Using the two names as seeds, the analyst extracted another previously unknown network. The analyst and a colleague identified one of the seed names as a person of interest in robberies involving multiple juveniles. Through cooperation and an SNA social diagram, they pieced together robberies not previously thought to be connected and identified a suspect involved in other robberies. The chart provided a source where they quickly, easily, and effectively could share observations with investigative personnel.



Social Network Diagrams



Social network diagrams have become a method for RPD to use social relationships among offenders and their associates. Renowned for its technological innovation and policing strategies, RPD has found SNA effective in facilitating better communication between crime analysts and investigators. SNA enabled the department to significantly increase crime clearance rates and reduce violence.



Prior to the SNA training, analysts conceptualized a series of “star” networks with an “ego” at the center and immediate connections radiating out. To understand the network, analysts identified the immediate connections of a person of interest. To identify one of the people connected to the original person of interest, a second ego network was constructed. In the end, the analyst faced a series of networks, leaving out the interconnections between them. Through training, analysts began to interpret the effectiveness of the larger network environment using diagrams as social maps to orient themselves and officers.



In the shooting case, the detective used the network analysis to apply pressure to the suspect by interviewing an associate whose relationship with the offender previously was unknown. Mounting social and financial pressures, ultimately, led the individual to surrender. In the convenience store robberies, an SNA diagram provided vital clues that allowed multiple analysts to share information and identify previously unknown connections between individuals, which led to a possible suspect. If SNA had been available to analysts in other jurisdictions, a connection may have been discovered earlier.



Analysis



The cases described illustrate the success of SNA in developing law enforcement strategies and interdiction techniques. The pilot project demonstrated how SNA can help answer sophisticated questions regarding motivations for a crime—an area previously underdeveloped in crime analysis processes.3 The research team was asked to determine why violence occurred among groups who previously were amicable. Using visual analysis and without any subject matter knowledge, investigators used SNA to reveal behavioral motivation rooted in complex interpersonal relationships. The project provided confirmation of the effectiveness of the current resource allocation of the gang unit and indicated new avenues of policing, which have the potential to produce a high return on investment.



These two cases produced actionable results, illustrating how SNA can facilitate a productive working relationship between crime analysts and detectives. The academic research on policing indicated that one of the biggest hurdles in establishing effective communication is finding a common language between the analytics of numbers and the immediate pressures of reality.4 Each case described illustrates how SNA and social network diagrams function as a common ground. Analysts used the charts visually to depict their analysis, which resonated with detectives because it reflected their reality. The analysts provided something new to the detectives, thus, aiding each investigation.



The visual and quantitative output of SNA helps solve institutional memory issues associated with analysts’ longevity and attrition, as well as new hires. By producing a current overview, SNA allows new analysts to grasp the present status of the network. It assists experienced analysts in maintaining an understanding of the network by chronicling growth and development as members and connections appear and disappear.



Law enforcement agencies, such as RPD, benefit from having access to structured, relational, and temporal data. Analysts reliably map changes in the network using an automated extraction process. Through this dynamic procedure, experienced analysts appear less likely to develop data analysis blind spots.



Conclusion



Law enforcement agencies have come a long way from pinpoint mapping. The technological advancements in recent years can provide personnel more confidence to handle complex crime problems confronting departments around the country. Social network analysis demonstrated its utility and effectiveness as a means of solving crimes or determining persons of interest and bridging the gap between crime analysts and police officers in the field. With the support of robust technology, SNA becomes reliable across time, data, analysts, and networks and quickly produces actionable results inside any operational law enforcement environment.



ENDTEXT



The authors commend and recognize the Richmond, Virginia, Police Department’s Crime Analysis Unit for its critical role and ongoing cooperation in the research and writing of this article.

FBI — Social Network Analysis: A Systematic Approach for Investigating
 
.
Could this technique be used to gain valuable suggestions or thoeries that could be utilized by these repsective agencies?
 
.
Could this technique be used to gain valuable suggestions or thoeries that could be utilized by these repsective agencies?

most likely will be used to crush the voices of opposition and those of not on the same page like some of our ministers demanding to ban Facebook ;)

But I will be more than happy if they did what you write.
 
.
Nothing of the sort, "upscale" will be giving detailed analysis reports, customer and peoplee response in a structured manner, analysis studies and conclusions drawn by analysts, debaters, experts and opinion makers.

This will give a fair idea to the organiization as to where its heading and what changes are expected of them from the people.

It also will give them multiple source data and feedback on each event or announcement made by them.
 
.

Pakistan Defence Latest Posts

Pakistan Affairs Latest Posts

Back
Top Bottom