How Search engine companies depend on data mining. (Basic Idea)

Introduction

From world wide web (www) we can get many type of information such as Page Link, web page, accessible document, images videos and also many type of content .So database continually increase. The WWW has added abundant of data and information transform into complex information. For the complex and large volume of information, it is not easy to find relevant information in a short time. In this regard problem has been resolved by data mining which is a process of extracting previously unknown data. However, data mining is a process by which previously unknown information and patterns are extracted from large quantity of data. I try to describe basic idea of search engine and data mining.

Search engine:

For a large volume of data on internet it is difficult to find and extract information for you. It has said that if you spend only one minute per page, 10 hour a day, it would take four and half year to explore only 1 million web pages. So for real need data mining is necessary. There are many search utilities such as google, bing, ask, AOL, webCrawler etc. Every search engine has large database.

A search engine database typically contains information such as

1.      Title of the page

2.      The url

3.      A short abstract of the content

4.      Keyword to help the search engine

Web sites are indexed, scored and ranks for different search engine. Ranking algorithms are work by web site usability and search frequency of keywords.

For Example: If 10 different user are search by “Data mining” text within 15 user. Other 5 user search by “Data mining and search engine” text. First 10 user also have interest of search engine related result. Here frequency of “Data mining” related web pages is increase. So next time when any one write “Data Mining” text for pick result then most browsed web site will show first. 

Data Mining:

Data mining extract related data for you from large database by use of KDD(Knowledge discovery in database) .

KDD can be :

1.      Database

2.      Relational database

3.      Structure database

4.      Unstructured database      

5.      Flat file

6.      Transactional database

7.      Object Oriented database

8.      Data Warehouse

9.      Multimedia database

10.  Time series database

You can use Association and clustering analysis in search engine algorithm to extract required result.

1.    Association Analysis:

Association analysis discovers the pattern that describes strongly associated features in data. For example: they who search by text “data mining” would most possibility to enjoy “data mining and search engine” related result.

2.    Cluster Analysis:

Cluster analysis seeks to find groups of closely related observations so that observations belong to same clusters are more similar to each other.

For example: Search result of data mining and data science may closely related.

Bibliography

[1] Mohammad Alhamami,Using Data mining to enhance web search engine. Ref: http://www.ehulool.com/using-data-mining-to-enhance-web-search-engines/

[2] Hillal Hadi Saleh, Mohammad Ala’a AL-Hamami, “A Proposed System to Improve Relevant Information Retrieval on the Web”, the 1st International Conference on Digital Communications and Computer Applications (DCCA2007), the Jordan University of Science and Technology, Irbid, Jordan.2007.
[3] Alaa H. AL-Hamami, Mohammad A. AL-Hamami, Soukaena H. Hashem, “Using Data Mining Confidence and Support for Privacy Preserving Secure Database”, Journal of Statistical Sciences, Volume 1, No. 1, Issued by Arab Institute for Training and Research in Statistics, July –December 2009.
[4] Smith J. R., and Chang S. F., “Visually Searching the Web for Content”, IEEE Multimedia Magazine, vol. 4, pp. 12-20, 1997.

[5] Pang- Ning Tan,Michael Steinbach,Vipin Kumar,”Introduction to data mining” 2006

Designing and developing new software programs

An effective plan can make software development easy, error free, better deliverable and accurate. If you read about database ACID (Atomic, Consistency, Isolated, Durable) then you can conceptually prepare your mind how you can create an effective plan for software design and development. For better project you need to create undoubtedly plan and executed that sequentially. As software development process is more complex than any other project, you need to develop in a perfect plan. Software development life cycle(SDLC) is basic for software development. Based on SDLC you should requirement gathering from client first. After that you should sequentially operate requirements analysis, design, program development, testing and finally deployed deliverable software. There are lots of method to maintain SDLC for cost effective and bug free deliverable such as water fall ,Incremental, rapid application development ,Agile ,Scrum, extreme programming etc. You need to choose one based on your project volume, clients type and your organization need.     

Requirement Analysis

Requirement Analysis (Short description for better remember):

In software development life cycle requirement analysis comes after requirement gathering process. Requirement analysis also called requirement engineering. Requirement analysis is a process which defined user needs for the software product. It must be quantifiable, detail and better understandable for both clients and product developers. In software engineering these feature details are called functional requirement. This process required for new software development, new feature adds within the existing system or modify an module.

Requirement analysis involves frequently communication with system users to determine their expectations. It always focuses on client need rather than mold users expectation. In this process analyst usually create a document called software requirement specification which contains full details of functionality. Also in all aspect it contains project development process from start to end.

Requirement analysis is a team effort contains hardware, software and human factors engineering both developers and Clients.

System Architecture for integration between bank software and organization accounting software

1.1 Introduction:

Every organization do their financial transaction through financial institution .They have accounts for collect bills/fees/distributor payments .Once financial institution receive any amount of money at these account’s they entered some information’s in their software .Each organization get the statement and manually posted to their accounting system. This kind of job indicates work redundancy. Also bank has multiple channel to deposit or withdraw money such as branch banking, Internet banking,mobile banking ATM etc.

This article will help software engineer to create integration system between bank system and organization accounting system.

1.2 Document Key word:

Organization accounting system: OAS

Bank system: FIS

DB: Database

MW: Middle Ware

VPN: Virtual Private Network

1.3 Scope:

The following scopes are currently available for this system.

-Bank will collect bill’s/Fees/Invoice.

-Send bill/fee/Invoice amount to organization database just in time.

For better understand and easy document writing I have assumed a organization named ORG and Bank named FI.

1.4 Overall Description

1.4.1 System Environment

                                    Fig1.1: System Environment

1.4.2 Steps for setup the integration system:

Step1:

            FI and organization MW(Middle Ware) server’s will be connect through VPN(Virtual Private Network) connection.

Step 2: Each MW server will connect with MW database.

Step 3: ORG will develop 3 web service method’s.

            a.Find

            b.PaymentConfirmation

            c.Reverse

Step 4: FI Need to access ORG web service.

Step 5: FI Need to develop a web service named SyncValidData which will post data to their MW DB to Main DB.

Or

 FI need to develop a DB Trigger named SyncValidData which will post data to MW DB to Main DB.

Step 6: ORG need to develop a web service named SyncValidData which will post data to MW DB to Main DB.

Or

ORG need to develop a DB Trigger named SyncValidData which will post data to MW DB to Main DB.

Step 7: ORG accounting system must insert process bill to its MW DB

1.5 Needed information

1.5.1 For FI:

1. Bill No

2. Bill Name

3. Total Amount

4. Vat

5.Status(Initiate/delivered/Reversed)

1.5.2 For ORG:

1. Transaction No

2. Transaction Date

3. 5.Status(delivered/Reversed)

1.6 Reference:

1. Web service

2. DB Trigger

3. VPN(Virtual Private Network)


My Others:
Web: http://techhunter047.com
Other Blog: http://ghorhishab.blogspot.com
Software: http://techhunter047.com/ghorhishab