background preloader

MapReduce

MapReduce
Overview[edit] MapReduce is a framework for processing parallelizable problems across huge datasets using a large number of computers (nodes), collectively referred to as a cluster (if all nodes are on the same local network and use similar hardware) or a grid (if the nodes are shared across geographically and administratively distributed systems, and use more heterogenous hardware). Processing can occur on data stored either in a filesystem (unstructured) or in a database (structured). MapReduce can take advantage of locality of data, processing it on or near the storage assets in order to reduce the distance over which it must be transmitted. "Map" step: Each worker node applies the "map()" function to the local data, and writes the output to a temporary storage. MapReduce allows for distributed processing of the map and reduction operations. Another way to look at MapReduce is as a 5-step parallel and distributed computation: Logical view[edit] Map(k1,v1) → list(k2,v2) Examples[edit]

Performance Computer performance is characterized by the amount of useful work accomplished by a computer system or computer network compared to the time and resources used. Depending on the context, good computer performance may involve one or more of the following: Technical and non-technical definitions[edit] The performance of any computer system can be evaluated in measurable, technical terms, using one or more of the metrics listed above. This way the performance can be - compared relative to other systems or the same system before/after changes - defined in absolute terms, e.g. for fulfilling a contractual obligation Whilst the above definition relates to a scientific, technical approach, the following definition given by Arnold Allen would be useful for a non-technical audience: The word performance in computer performance means the same thing that performance means in other contexts, that is, it means "How well is the computer doing the work it is supposed to do?" Performance engineering[edit]

Secure coding Securing coding is the practice of developing computer software in a way that guards against the accidental introduction of security vulnerabilities. Defects, bugs and logic flaws are consistently the primary cause of commonly exploited software vulnerabilities. Through the analysis of thousands of reported vulnerabilities, security professionals have discovered that most vulnerabilities stem from a relatively small number of common software programming errors. See also[edit] References[edit]

Security Security is the degree of resistance to, or protection from, harm. It applies to any vulnerable and valuable asset, such as a person, dwelling, community, nation, or organization. As noted by the Institute for Security and Open Methodologies (ISECOM) in the OSSTMM 3, security provides "a form of protection where a separation is created between the assets and the threat." Perceived security compared to real security[edit] Perception of security may be poorly mapped to measureable objective security. Security theater is a critical term for deployment of measures primarily aimed at raising subjective security without a genuine or commensurate concern for the effects of that measure on objective security. Perception of security can increase objective security when it affects or deters malicious behavior, as with visual signs of security protections, such as video surveillance, alarm systems in a home, or an anti-theft system in a car such as a vehicle tracking system or warning sign. Safety

NLM APIs An Application Programming Interface (API) is a set of routines that an application uses to request and carry out lower-level services performed by a computer's operating system. For computers running a graphical user interface, an API manages an application's windows, icons, menus, and dialog boxes. We invite you to develop computer and mobile applications using National Library of Medicine (NLM) resources. We request that any application that makes use of NLM data include the following statement: "This product uses publicly available data from the U.S. Developers may not use the NLM name and/or logo in conjunction with their applications. Supported NLM APIs are listed below, along with information about any additional terms and conditions of use. We encourage comments and recommendations for further API development to NLM customer service. More data resources are available at the Databases, Resources & APIs page. Upcoming Events Event Archive Discussion slides are available for download.

Service Component Architecture Service Component Architecture (SCA) is a software technology created by major software vendors including IBM, Oracle and TIBCO. SCA provides a model for composing applications that follow Service-Oriented Architecture principles.[1] The technology encompasses a wide range of disparate technologies and as such is specified in various independent specifications in order to maintain programming language and application environment neutrality.[1] Partners[edit] Partner vendors include: Supporters[edit] In addition to the partners above, the SCA community has a significant set of formal supporters.[3] The Supporters Program remains open for any interested vendor, ISV, customer or user of the SCA technology to contribute to its evolution. Definition[edit] On March 21, 2007, the OSOA Collaboration released the V1.0 level of specification.[4] The specifications specify that an application designed with SCA should have the following advantages: Further analysis[edit] Advantages: Disadvantages: [edit]

WebSphere Application Server V7 Feature Pack for Service Component Architecture - FAQ What is Service Component Architecture? Service Component Architecture (SCA) was conceived through industry collaboration to provide a language-neutral programming model for building applications based on Service Oriented Architecture. First published in 2005, the Open SOA Collaboration (osoa.org) finalized the SCA 1.0 specification and submitted to OASIS for standardization in March 2007. The SCA programming model benefits architectures where business function is partitioned as a set of services. Is SCA standardized? While made up of many industry leading vendors, the Open SOA Collaboration is not intended to be a standards body. What vendors are participating in the Open SOA Collaboration and the OASIS Open CSA standards body? The list of participating vendors and supporters is always increasing. OASIS Open CSA participants are listed here: Is Open CSA a new name for SCA? No, it is not. Are there any open source communities that support SCA? Yes. No.

OASIS (organization) The Organization for the Advancement of Structured Information Standards (OASIS) is a global consortium that drives the development, convergence, and adoption of e-business and web service standards. With its headquarters in the United States, members of the consortium decide how and what work is undertaken through an open, democratic process.[1] Technical work is being carried out under the following categories: Web Services, e-Commerce, Security, Law & Government, Supply Chain, Computing Management, Application Focus, Document-Centric, XML Processing, Conformance/Interop, and Industry Domains. OASIS was first formed as SGML Open in 1993 as a trade association of SGML tool vendors to cooperatively promote the adoption of SGML through mainly educational activities, though some amount of technical activity was also pursued including an update of the CALS Table Model specification and specifications for fragment interchange and entity management.

OASIS OASIS SOA Reference Model A reference model in systems, enterprise, and software engineering is an abstract framework for understanding significant relationships among the entities of some environment, and for the development of consistent standards or specifications supporting that environment. A reference model is based on a small number of unifying concepts and may be used as a basis for education and explaining standards to a non-specialist. A reference model is not directly tied to any standards, technologies or other concrete implementation details, but it does seek to provide a common semantics that can be used unambiguously across and between different implementations. Description[edit] History[edit] The OASIS SOA Reference Model, is a product of the OASIS SOA Reference Model (SOA-RM) Technical Committee (TC).[1] Prior to this initiative, no standard definition of SOA had existed. Current status[edit] Future Work[edit] Principal Concepts[edit] OASIS definition of SOA[edit] Service[edit] SOA Example[edit]

Service-oriented architecture See also the client-server model, a progenitor concept A Service-Oriented Architecture (SOA) is a design pattern in which software/application components provide services to other software/application components via a protocol, typically over a network and in a loosely-coupled way. The principles of service-orientation are independent of any vendor, product or technology.[1] A service is a self-contained unit of functionality, such as retrieving an online bank statement.[2] By that definition, a service is a discretely invokable operation. However, in the Web Services Definition Language (WSDL), a service is an interface definition that may list several discrete services/operations. Services can be combined to provide the complete functionality of a large software application.[3] A SOA makes it easier for software components on computers connected over a network to cooperate. Definitions[edit] The Open Group's definition is: Overview[edit] SOA framework[edit] Design concept[edit]

Open SOA Collaboration Last month an alliance of leading vendors announced progress on specifications to define a language-neutral programming model for application development in SOA environments. They call this specification Open SOA Collaboration. In essence, they are proposing a new standard to create and manage IT, making the process of integrating different third-party SOA technologies "less onerous," they say. Or, we can call this a standard way of delivering services, making it easier to work and play well together. So, who's in the gang? This group has concentrated its efforts on two projects - service component architecture (SCA) and service data objects (SDO). SCA is looking to provide a model for creating service components in a wide range of languages and a model for assembling service components in a business solution. SCA encourages an SOA organization of business application code based on components that implement business logic. New?

SpringSource.org

Related: