ICT705 Data and System Integration Task 2 and Tutor Proposal

USC


Individual assignment

Assessment No: 2

ICT705|Data and System Integration

Our Real
uni-icon
Student’s Score cards


ICT705  Data and System Integration  Task 2 and Tutor Proposal

ICT705 | Data and System Integration | Task 2


Task

Background: Queensland Fire and Emergency Services

Given the recent outbreak of bush fires across the country the Queensland Fire and Emergency Services (QFES) have noted deficiencies in their current systems. After evaluating various platforms, the QFES has decided to adopt a Service Oriented Architecture (SOA) for its future IT infrastructure.

However, they would like to gain a deeper understanding of the technologies and have asked for a demonstration of these technologies along with a brief explanation of the concepts and principles of how it works.

Specifically, they have asked you to provide a demonstration web page in which the user will input a region from a drop box and click a button to display a google map with the locations of fire stations in the region.

Data for this assignment is based on data collected from Queensland Government websites, specifically:

Station_Locations.xml was constructed from RuralFireStations.csv

Queensland Fire and Emergency Services - Rural Fire Stations dataset by Queensland Fire and Emergency Services from Open Data Portal (CC BY 4.0)

Fire_Stations.csv was constructed from data contained on web pages

Queensland Fire and Emergency Services – Regions by Queensland Fire and Emergency Services (CC BY 4.0)

Data from these two sources was downloaded and underwent some significant manipulation and cleaning to produce the input files for your assignment, as working with the original data sources was deemed too difficult for the course.

Station_Regions.csv was constructed consistent with the above data sources to help you complete your assignment.

For those students looking for a challenge, you should try writing code that automatically extracts and merges data from the original two sources, to experience a real-life example of the problems of data integration even within one organisation (unfortunately no extra marks can be given for this).

Assignment Tasks

This assignment consists of two deliverables, being: 1. Demonstration code 50%, and 2. Report 50%.

  • Demonstration Code - All code relating to this assignment should be contained within a folder named: Task 2 – Your Name Student Number, the folder is then to be zipped and uploaded to blackboard. (note replace “Your Name” and “Student Number” above with your actual name and student
  • A report in Microsoft Word Format. The report must be uploaded

Deliverable 1 - Demonstration Code

Important Note: For ease of demonstration, your code must be self-contained. In addition to using Standards based HTML and Python, PETL, Bottle and Parser are the only additional frameworks you should need. If you use others they must be provided and must not require installation.

To demonstrate your code, you have been provided with three data sources compiled with data from QFES. These are:

  • “Fire_Stations.csv” contains the information about the fire stations,
  • “Station_Locations.xml” contains the location coordinates for each fire station
  • “Station_Regions.csv” contains a list of regions

You have also been supplied with two additional files, that you can use to check the correctness of your implementation, or if you are unable to complete some tasks:

  • “Fire_Stations (ASCII).csv” contains data from Fire_Stations.csv with non-printable characters removed
  • “Fire_Station_Locations (Solution).csv” is the required output for Task 1
Task 1 - Data integration demo.

You are to submit a python script named “data_integration.py”. Your script should clean and merge data into one CSV file named “Fire_Station_Locations.csv” with the following fields (attributes):

Field Name

Mapping

RegionID

Station_Regions.RegionID

Station Number

Fire_Stations.Stn Number

Station Name

Fire_Stations.Fire Station

Station Type

Fire_Stations.Stn Type

Street Address

Fire_Stations. Address

Phone Number

Fire_Stations.Phone Number

Fax Number

Fire_Stations.Fax Number

E-Mail

New Calculated Field

Lat

Station_Locations.Lat

Lon

Station_Locations.Long

Table 1 - Field Mapping

Task 1.1 Remove non-printable characters

The Fire_Stations.csv file contains non-printable Unicode characters, in order to perform other cleansing tasks and merging of data you’ll first need to clean these characters from the data.

Note: You have been provided with a file “Fire_Stations (ASCII).csv”, if you are unable to complete this task you may use this file to complete remaining tasks.

Task 1.2 Create E-Mail

An email address for each station needs to be constructed from its name: e.g. the address for Acacia Ridge Fire Station will be enquiry@acaciaridge.qfes.gov.au

Task 1.3 Merge Data

You are to merge data from the three sources into “Fire_Station_Locations.csv” with fields as shown in Table 1 - Field Mapping.

Task 1.4 Correct Field Names

Ensure your field names match the fields as shown in Table 1 - Field Mapping .

Task 1.5 Correct Field Order

Ensure the order of fields matches the order in Table 1 - Field Mapping

Task 1.6 Sort Data

Data must be sorted firstly on “RegionID” and then on “Station Number”

Note: Should you be unable to fully complete the above tasks, you have been supplied with

a file “Fire_Station_Locations (Solution).csv” that you can use for Task 2.

Task 1.7 Clean and Separate Street Address (Optional

Separate data in the Address field into: “Street Address”, “Suburb” & “Post Code” fields.

Create a “State” field with a value of “Qld” for each record

Replace abbreviations for street type (eg. Rd is used instead of Road, Crt instead of

Note: This task is optional should you choose to attempt it any marks scored on this task will be added to your overall mark for the assignment up to 100%, that is you can’t score more than 100% for the assignment.

Task 2 – RESTful Web Service Demo

You are to submit a python script named “data_services.py”.

For tasks 2.1 to 2.2 Your script should build a RESTful web service that supports a

‘getstations’ query from the client browser similar to “/getstations?regionid=x”. The server Web Service will return a JSON table with the following fields (attributes): Station Number, Station Name, Street Address, Phone Number, Email, Lat, Lon. Data is to come from the “Fire_Station_Locations.csv” you built in task 1 (or, “Fire_Station_Locations (Solution).csv” if you were unable to fully complete task 1).

Task 2.1 Return stations for a region

The server will return a table of records (tuples) of fire stations in the region passed to it.

Task 2.2 Return all fire station

In the advent of a call to “getstations?regionid=0”, the server will return a list of all fire stations.

Task 2.3 List of Regions

To populate your drop-down list in task 3.1 you will need to return a JSON table of all records from “Station_Regions.csv” with the following fields (attributes): RegionID, Region Name. The server should respond to a “getregions” request.

Task 3 Mashup Demo

You are to submit a HTML file “stations_map.html” and optionally a CSS file named

“stations_map.css”. For ease of marking all JavaScript must be contained in your HTML file.

Task 3.1 Form Layout

You are to create an input form that accepts user input for a station region. The input for the station region should be a drop-down list with the first item being “All Regions” with a value = 0 remaining items should be created dynamically from a call to getregions from task

2.3. This input should be accompanied with a “Display Stations” button and appropriate descriptors / information on the operation of the web page.

Task 3.2 Google Map

Clicking the “Display Stations” button will trigger a “getstations” web service to retrieve a table of stations matching the region with attributes described in task 2. From this data you are to display a Google map marking the position of the returned stations.

Task 3.3 Label Markers

Markers on the map should be labelled with the “Station Number”

Figure 1 - Markers labelled with Station Numbers

Task 3.4 Centred & Zoomed

The map should be centred on the region and zoomed to an appropriate level to show all fire stations in the region selected.

Task 3.5 Information Window Demographics

Upon clicking a marker bubble on the map, the user will be displayed an information window containing: Station, Address, Phone and a clickable email address.

Figure 2 - Demographic Window

Task 3.6 Information window email

Clicking an email address link in the information window should launch the user’s email

application and automatically insert the subject line “Enquiry”.

Deliverable 2 - Report

You are required to write a report explaining the key concepts around the design and implementation of your demonstration system. You are also required to explain how to run your code. Finally, you are to include all code submitted in the appendices to the report.

Executive Summary, Introduction, Conclusion & Recommendations

Please follow the conventions detailed in:

Summers, J. & Smith, B., 2014, Communication Skills Handbook, 4th Ed, Wiley, Australia.

2.1 Data Merging and Cleaning

Provide a scholarly definition of data merging and data cleaning. What issues did you face and how did you deal with these?

What recommendations do you have for QFES regarding data integration and why?

2.2 RESTful Web Services

Provide a scholarly definition of REST.

How was REST incorporated in your demo?

What problems can you identify with the design of your demo?

2.3 Mashups

Provide a scholarly definition of Mashups.

How were mashups incorporated into your demo?

How could you improve the system to provide a more efficient mashup?

3.0 Demo Running Instructions

Given your client QFES has limited technical knowledge, you are to provide instructions for getting the ZIP file you supplied (the one you uploaded to BlackBoard) up and running. You should presume that they have a new workstation with a clean installation of Windows to test your demo.

Report Format

The report must be formatted using the following guidelines:

  • Paragraph text 12-point Calibri single line spacing
  • Code Text 12-point Courier new single line spacing
  • Headings Arial in an appropriate type size
  • Margins 2.5cm on all sides
  • Title Page is not to contain headers, footers, or page numbering
  • Header is to display the Report Title (excluding the title page)
  • Footer to display your name and the page number (excluding the title page)
  • Title Page – Must not contain headers or Include your name as the report’s

author.

  • Page numbering
    • Executive summary to the last page of Table of Figures / Table of Tables to use roman numerals (i, ii, iii, iv)
    • Introduction and onwards to use conventional numerals (1, 2, 3, 4) starting at page 1 from the introduction
  • The report is to be created as a single Microsoft Word document (version 2007, 2010, 2013, 2016, or Office 365). No other format is acceptable and doing so will result in the deduction of
  • The report must use the built-in features of Microsoft Word to create page numbers, section numbers, citations, cross referencing of sections, figures and tables, table of contents, table of figures, table of tables, and

Why invest in our services?

Only High Quality
Optimum quality

Our assignment help team is trained to provide you high quality writing services.

Reasonable Price of Each
High scores

High scores achieved by our students is a portrayal of our high quality online assignment help

Privacy and Security
Multiple reach

You can place your assignment order through 4 easy modes of communication

Order Now

TUTOR PURPOSAL


Here is how, we will write your assignment

tutor-proposal

We write your assignment in the following proposed way and we make sure that it is strictly in accordance to your assignment guidelines and each section of the assignment is written with respect to its marking criteria/ rubric.

TUTOR PROPOSAL FOR : ICT705|Individual assignment

ICT705 Data and System Integration  Task 2