Automating Real-World Tasks with Python https://www.coursera.org/learn/automating-real-world-tasks-python/home/module/1

 https://www.coursera.org/learn/automating-real-world-tasks-python/home/module/1

Module 1

Course introduction

Module 1

Welcome to the Course!

Module 1

Module 1 Introduction 

understand how to read Python APIs, and learn how to use the Python Imaging Library (PIL)  

Module 1

Distributed systems

Key takeaways

Distributed systems are crucial in various applications, but require careful design and management to address their complexities and potential challenges.  

• Definition of distributed systems - A distributed system is a collection of software components that collaborate across separate servers or nodes, often using a shared network. These systems aim to eliminate bottlenecks and single points of failure by distributing tasks and functions across multiple components.

• Characteristics of distributed systems - Distributed computing systems exhibit several key characteristics, including resource sharing, error detection, transparency, simultaneous processing, scalability, and heterogeneity.

• Advantages and disadvantages - Distributed systems offer advantages such as flexibility, handling large volumes of traffic, redundancy of nodes, and fault tolerance. However, they also come with disadvantages like increased complexity, the need for extra work to locate components, potential introduction of new problems and delays due to network issues, and increased costs associated with scalability.

Module 1 NALSD

NALSD is Non Abstract Large System Design by google.

Aims to give SREs System Reliability Engineer the ablity

            1. To assess

            2. To Design

            3. To Evaluate

Large Systems.

Key characteristics of NALSD

        NALSD requires DevOps team to think of 

                1. Scale

                2. Resilience (flexiblity)

during the process design

    NALSD is divided into two Phases

        1. Technical Design

        2. Scalling up

Technical Design:

        the team tries to answer two questions about the proposed design:

• Q 1:

• Is it possible?  

  Will the design even work?

• Q 2:

• Can we do better?  

  Can we make it faster, simpler, or cheaper?

Phase 2: Scaling up

        How will the system be able to accommodate a random increase in users?

            1. Is it feasible?  

• Will it work at scale?  

  Is it cost-effective? 

• 2. Is it resilient?  

  What happens if the database goes down?   

• 3. Can we do better?  

  Are there changes or  

  additions that we need to make?

Three key goals of NALSD

       1.  The first is proper capacity planning.

       2. The second is component isolation.

       3. The final goal is graceful degradation

Google’s NALSD Workbook 

        The NALSD Workbook was created by Google's SRE team. It         is designed to help engineers and developers with the design and     architecture of large-scale, reliable systems.

https://static.googleusercontent.com/media/sre.google/en//static/pdf/nalsd-workbook-letter.pdf

Module 1

Built-In Libraries vs. External Libraries

Python Package Index (https://pypi.org).

PIL does not support Python 3. Fortunately, there's a current fork of PIL called Pillow,

user@ubuntu:~$ sudo apt install python3-pil

Reading package lists... Done

Building dependency tree     

(...)

Unpacking python3-pil:amd64 (4.3.0-2) ...

Setting up python3-pil:amd64 (4.3.0-2) ...

--------------------------------------------------------------

$ pip3 install pillow

Collecting pillow

  Downloading https://files.pythonhosted.org/packages/85/28/2c72ba965b52884a0bd71e419761fc162763dc2e5d9bec2f3b1949f7272a/Pillow-6.2.1-cp37-cp37m-macosx_10_6_intel.whl (3.9MB)

     |████████████████████████████████| 3.9MB 1.7MB/s

Installing collected packages: pillow

Successfully installed pillow-6.2.1

.

Module 1

What is an API?

 Application Programming Interfaces (APIs) help different pieces of software talk to each other.

These libraries provide APIs in the form of external or public functions, classes, and methods that other code can use to get their job done without having to create a lot of repeated code.

Module 1

How to Make Sense of an API?

pip install PIL is for python 2

$ pip install pillow

$ python

>>> from PIL import Image

>>> im=Image.open('d:bridge.jpg')

>>> im.show()

>>> im.rotate(45).show()

Pillow's full documentation is published here.

and they’ve also written a handbook

with tutorials for you to get familiar with the library's API

Module 1 Generate and manage containers

Generating image containers

            To build a container image, you create a Dockerfile, which contains step-by-step instructions for Docker to package your application along with all its dependencies.

https://docs.docker.com/get-started/02_our_app/

Choose a base image

One important step is deciding which base image to use.

Here are some of the most popular base images:

• Debian and Ubuntu: containers that boot into a full-featured, general Linux environment 

• Alpine Linux: a stripped-down image designed to result in small, fast containers

• Python: great for running Python apps

Create a Dockerfile

Here’s a sample Dockerfile for a Python web app that uses Flask and SQLAlchemy:

7

4

5

6

1

2

3

CMD [ "flask", "run", "--host=0.0.0.0", "--port=4000"]

RUN pip install -r requirements.txt

COPY . .

EXPOSE 4000

FROM python:3.9

WORKDIR /app

COPY requirements.txt ./

requirements.txt file. Here’s a minimal one that just installs Flask, SQLAlchemy, and the PyMysql driver:

    1    flask

    2    pymysql

    3    Flask-SQAIchemy

Module 1

How to Use PIL for Working With Images

from PIL import Image

im = Image.open("example.jpg")

new_im = im.resize((640,480))

new_im.save("example_resized.jpg")

----------------------------------------------------------------------------------------------

from PIL import Image

im = Image.open("example.jpg")

new_im = im.rotate(90)

new_im.save("example_rotated.jpg")

-----------------------------------------------

from PIL import Image

im = Image.open("example.jpg")

im.rotate(180).resize((640,480)).save("flipped_and_resized.jpg")

There's a ton more that you can do with the PIL library. Have a look at the docs and try it on your computer!

Module 1

Project Problem Statement

However, the contractor has delivered the final designs and they’re in the wrong format, rotated 90° and too large.

Write Python program to correct it. using pillow library

Module 1

Glossary terms from course 6, module 1

Distributed systems:

                Also referred to as distributed computing or distributed databases, utilize different nodes to interact and synchronize over a shared network

Docstrings: Documentation that lives alongside the code

NALSD (Non-Abstract Large System Design):

                A discipline and process introduced by Google, primarily aimed at empowering site reliability engineers (SREs) to

                    assess,

                    design, and

                    evaluate

large-scale systems

Naming conventions:

                Functions, classes and methods with naming conventions to understand what to expect from them

 Module 1

Qwiklabs assessment: Scale and convert images using PIL

What you’ll do

        Use the Python Imaging Library to do the following to a batch of images:

• 1. Open an image

• 2. Rotate an image

• 3. Resize an image

• 4. Save an image in a specific format in a separate directory

Pillow tutorial website is 

https://pillow.readthedocs.io/en/stable/handbook/tutorial.html

--------- Starting of  myscript.py------------------------

#! /usr/bin/env python3

import os

from PIL import Image

for infile in os.listdir('images'):

    if 'ic_' in infile:

        im=Image.open('images/'+infile)

        newimage=im.rotate(-90).resize((128,128)).convert('RGB')

        outfile = '/opt/icons/'+infile+'.jpeg'

        newimage.save(outfile,'JPEG')

--------- The End of  myscript.py------------------------

Module 2 Introduction

You'll first learn 

        1. how you can use different text formats to store data in text                 files, 

        2. retrieve it, and even 

        3. transmit it over the internet.

      4. How we can get our code to interact with services running on         different computers using a module called Python                                 Requests.

NOTE:

        best way to get comfortable with all these modules and                     libraries is to come up with your own examples and practice             writing scripts on your local computer!

Module 2 Web Applications and Services

            

https://www.youtube.com/watch?v=ukIE6Af4v5k

 by Durgesh

Lots of web applications also have APIs that you can use from your scripts!

Webservices:

            Web applications that have an API are also known as web services.

API call:            

        You can use your program to send a message known as an API call to the web service.

API endpoint.

            The part of the program that listens on the network for API calls is called an API endpoint.

https://www.youtube.com/watch?v=ukIE6Af4v5k

by durgesh  at 6:07  

Module 2 RESTful APIs

RESTful APIs use HTTP requests to perform CRUD (create, read, update, delete) operations on resources.

RESTful methods

RESTful APIs work by associating methods (functions) with resources. Some of the most commonly used HTTP request methods are:

• GET:

• Requests using GET should only retrieve data.

• HEAD:

• The HEAD method asks for a response identical to a GET request, but without the response body.

• POST:

• The POST method submits an entity to the specified resource, often causing a change in state or side effects on the (remote) server.

• PUT:

• The PUT method replaces all current representations of the target resource with the request payload.

these are HTTP response codes.

N.B. NOTE: 

        RESTful APIs almost always use JSON, 

        but RESTful APIs can also be used to send and receive files, as         well as stream data. 

JSON:

          JSON serves as the standard payload format for                     transmitting             data  

                With its key-value pairs, JSON is both human readable                 and machine friendly, making it a popular choice for web-based         APIs.

Key takeaways

        RESTful APIs are a fundamental and versatile part of web development.

        Knowing how to design, consume, and work with them is essential for building modern web applications, ensuring they can communicate effectively with other services.

Module 2

What is REST architecture?

        REST stands for 

                    1. Representational 

                    2. State 

                    3. Transfer.

        REST Architecture is use for 

                    1. designing networked applications and 

                    2. web services.

           REST was invented as a standard way for clients and servers to communicate with each other over the internet. 

Constraints with the REST architecture

6 Costraints with REST architecture

The six constraints are:

• 
  

• Uniform interface -

• There should be consistent methods for clients to access and change resources on the server using standard HTTP (Hypertext Transfer Protocol) conventions.  

• 
  

• Stateless -

• Every piece of information the server requires to process the request should be within the request. There shouldn't be any leftover information on the server between requests.

• 
  

• Cacheable -

• Every server response should indicate whether the data can be cached on the client and the length of time is needed to cache the data.

• 
  

• Client-server -

• The client and server can evolve independently. The REST interface serves as a “contract” between them.

• 
  

• Layered system -

• An application should be split into layers. Each layer of the application handles a particular concern (data access, business logic, presentation, etc) and acts independently from the other layers.

• 
  

• Code on demand (optional) -

• Servers can also provide code to be executed on the client. This enables the client to change its behavior dynamically.

HTTP protocol with REST

        This design enables clients and servers to communicate over the public internet using standard HTTP conventions. 

        Companies will often publish their REST API (Application Programing Interface) so that developers can make use of it

        The standard HTTP features: 
                verbs, 
                headers, and 

                data payloads.

        HTTP allows clients to 

                GET, 

                PUT, and 

                DELETE resources. 

        Clients can also POST queries with complex data, such as performing a search or transferring money between accounts

        The PATCH verb allows clients to update a resource by just sending what has changed.

        In REST API client send Headers might include 

                authentication, 

                enable optional features, or 

                allow the client to request that the server send data in specific formats (e.g. JavaScript Object Notation, JSON, or eXtensible Markup Language).

to create REST APIs in Python, check out this link here.

further information for REST APIs with GCP, click on this link here. 

Module 2

Using REST APIs to access web data

Here are the key steps to access web data using RESTful APIs:

1. 
   

2. Identify the API endpoint:

   The endpoint is the URL that you will send your HTTP request to.

    

3. Select the appropriate HTTP METHOD: 

   1. POST: Create a new resource. 

      PUT: Update an existing resource or create it if it doesn't exist (replace the entire resource). 

      PATCH: Partially update an existing resource. 

      DELETE: Remove a resource. 

4. GET: Retrieve data from the resource. 

5. 
   

6. Set up request headers:

   Common headers include   

   1. authentication tokens (e.g., API keys or OAuth tokens),  

   2. content type, and  

   3. accept headers (indicating the desired response format, such as JSON or XML).

7. 
   

8. Prepare the request body:

   For HTTP methods like POST and PUT, you may need to prepare a request body containing data to be sent to the server. The format of the request body depends on the API's documentation.

9. 
   

10. Send the HTTP request:

    Use a programming language or tool (e.g.,  

    1. Python's requests library,  

    2. JavaScript's Fetch API, or  

    3. specialized API client libraries)  

    to send the HTTP request to the API endpoint. Include the chosen HTTP method, headers, and request body as appropriate.

     

11. Receive the HTTP response:

    The server will process your request and respond with an HTTP response. This response will include:

    1. 1. Status code: Indicates the outcome of the request (e.g., 200 for success, 404 for not found, 500 for server error)

    2. 2. Response headers: Contain metadata about the response

    3. 3. Response body: Contains the requested data, often in a structured format like JSON or XML

12. 
    

13. Handle the response:

    1. Parse the response body to extract the data you need. The format will depend on the API's documentation and the content type header (usually JSON or XML). 

    2. Check the status code to determine  

       1. if the request was successful or  

       2. if an error occurred.

    3. Handle errors gracefully by examining the response body or status code and providing appropriate feedback to the user. 

        

14. Implement pagination and filtering (optional):

    If the API supports pagination or filtering, you can include query parameters in the URL to request specific subsets of data or control the number of records returned.

     

15. Authentication and authorization:
    

16. Error handling:

17. Implement error-handling logic to handle potential issues, such as network errors, invalid responses, or HTTP status codes indicating errors (e.g., 4xx and 5xx codes). Provide informative error messages to the user.

18. Rate limiting (if applicable): Respect any rate limits imposed by the API to prevent excessive requests. Implement rate-limiting strategies on your end to ensure you don't exceed the allowed request rate.

19. Repeat as needed: If you need to access more data or perform additional actions, repeat the steps with the appropriate API endpoints, methods, and parameters.

By following these steps, you can effectively access web data using RESTful APIs and integrate that data into your web applications or services. It's essential to refer to the API's documentation for specific details on endpoint URLs, request formats, authentication, and other requirements.

Module 2

Python tools for REST APIs

Python REST API framework are

            1. Tool Kits

            2. Software libraries

and offers 

            1. functions and 

            2. tools 

that needed to built RESTFull APIs.

Two categories of API tools

            1. Client Side

            2. Server Side

Client-side

        REST API client uses low-level built-in urllib library

Requests

        it’s frequently chosen by developers. There are newer derivatives of Requests that are gaining popularity, like HTTPX and AIOHTTP. For a comparison of these three, see HTTPX vs Requests vs AIOHTTP. 

To learn more about Requests, read our comprehensive guide to Python Requests Library.

PycURL

            For advance  developer, you might want to try a client library like PycURL, which offers concurrent connections and can be much faster than Requests. Complicated and hard to learn.

SERVER-side

The most popular REST API server frameworks are 

            1. Flask, 

            2. Django, and 

            3. FastAPI.

Flask

            Flask is a flexible and comfortable web development framework that is easy to set up and simple to use, which makes it good for beginners. 

Uber, Pinterest, and Twilio were all built using the Flask framework.

Django

            Django is one of the most popular frameworks worldwide. It’s a free, open-source Python framework designed for building websites of any size, with any traffic needs. 

 it makes use of reusable code. 

Django is also more secure than Flask, 

 Developers may find Django slow because of the need to employ reusable modules and the need to check compatibility against previous versions. 

YouTube, Instagram, Spotify, and DropBox were built with Django.

FastAPI

                high-performing web framework for building web APIs in Python, and it includes hints similar to those in Python. The main downside of FastAPI is how new it is.

Netflix uses FastAPI internally. 

For more on comparing these three REST API frameworks, see “Choosing between Django, Flask, and FastAPI” and “Top Python REST API Frameworks in 2023.”

Module 2

What is Flask?

Flask is a Python library that makes it easier to 

            1. create web applications and 

            2. REST (Representational State Transfer) web services

                   PUT, GET, PUBLISH APIs

Why use Flask?

 Developers have the freedom to select their favorite 

            1. design pattern, 

            2. database, 

            3. plugins, and 

            other features using Flask.

Flask is great for rapid development,

Flask supports building REST API servers and microservices.

Flask's extensive options of extensions and libraries, such as 

            1. Flask-RESTful and 

            2. Flask-SQLAlchemy,

Flask has built-in support for debugging and unit testing. 

Integrating unit testing and a debugger enables quick debugging and development

An alternative to Flask

Flask is simple, but powerful. 

flexibility allows developers to choose the tools and libraries that best suit their projects’ needs.

It is often compared to Django, which is considered a much more heavyweight framework. Django requires more setup to start building your app, whereas Flask requires very little.

For more information on Django and how it compares to Flask, click here.

It's important to note that Flask's simplicity and flexibility may require developers to make more decisions about project structure and components than would more opinionated (restricted)  frameworks as in Django.

Popular Flask plugins 

Flask WTF:

                Create and process web forms

• Flask RESTful: Create REST API services

• 
  

• Flask login:  Handles user authentication and identity

• 
  

• Flask debug toolbar:

• Provides a handy browser toolbar to help you debug your Flask app

Flask's extensive collection of plugins can significantly expedite development and enhance your application's functionality

 These plugins cover a wide range of features, 

            1. from authentication and 

            2. form handling to 

            3. database integration and 

            4. caching, allowing you to tap into a large selection of pre-built solutions.

Key takeaways

• Flexibility and freedom:

• Flask's simplistic and minimalist design offers developers the freedom to select their preferred design patterns, databases, and plugins. This flexibility allows them to tailor their web applications according to their projects’ specific requirements. 

   

• Rapid development and scalability:

• Flask’s support for WSGI templates enhances scalability and flexibility, making it suitable for building REST API servers and microservices. On the other hand, Django, with its comprehensive feature set, can be slower to set up initially, but may offer advantages in terms of built-in functionality for larger, more complex projects.

• 
  

• Trust of large corporations:

• MIT, Reddit, Uber, Lyft, Zillow, Patreon, and Netflix, to build their applications.

Module 2

How to use Flask

Flask is a lightweight web framework used in Python. 

Its ease of use for beginners

Module 2

Data Serialization

            If you have two programs that need to communicate with each other, how do you get that data from one place to another? We're going to talk about two aspects of that problem: 

            1. what to send, and    (data, pictures, ) 

            2. how to send it.  (protocol, )

Data serialization 

            is the process of taking an in-memory data structure, like a Python object, and turning it into something that can be stored on disk or transmitted across a network

            Later, the file can be read, or the network transmission can be received by another program and turned back into an object again. Turning the serialized object back into an in-memory object is called deserialization.

Data serialization is extremely useful for communicating with web service

A web service's API endpoint takes messages in a specific format, containing specific data.

 CSV examples. 

name,username,phone,department,role

Sabrina Green,sgreen,802-867-5309,IT Infrastructure,System Administrator

Eli Jones,ejones,684-3481127,IT Infrastructure,IT specialist

we could turn this information into a list of dictionaries.

people = [

    {

        "name": "Sabrina Green",

        "username": "sgreen",

        "phone": "802-867-5309",

        "department": "IT Infrastructure",

        "role": "Systems Administrator"

    },

    {

        "name": "Eli Jones",

        "username": "ejones",

        "phone": "684-348-1127",

        "department": "IT Infrastructure",

        "role": "IT Specialist"

    },

]

-------------------------------------------------------------------

let's say we want to record more than one phone number for each person

 

people = [

    {

        "name": "Sabrina Green" ,

        "username": "sgreen",

        "phone": {

            "office": "802-867-5309",

            "cell": "802-867-5310"

        },

        "department": "IT Infrastructure",

        "role": "Systems Administrator"

    },

    {

        "name": "Eli Jones",

        "username": "ejones",

        "phone": {

            "office": "684-348-1127"

        },

        "department": "IT Infrastructure",

        "role": "IT Specialist"

    },

]

-------------------------------------------------------------------------------

Module 2

Data Serialization Formats

JSON (JavaScript Object Notation)

            is the serialization format that we'll use the most in this course. json module to convert our people list of dictionaries into JSON format

[dict1,dict2,dict3]

[{

    "name":"SAEED HASSAN",

    "username":"saeedhisbani",

    "phones":{"home":03335646544};

    },

{

    },

{

    } ]

----------------------------------------

import json

with open('people.json', 'w') as people_json:

    # people=[{"name":"Saeed Hassan",..,..,},{},{}]

    json.dump(people, people_json, indent=2)

-----------------------------------------

This code uses the json.dump() function to serialize the people object into a JSON file. The contents of the file will look something like this:

[

  {

    "name": "Sabrina Green",

    "username": "sgreen",

    "phone": {

      "office": "802-867-5309",

      "cell": "802-867-5310"

    },

    "department": "IT Infrastructure",

    "role": "Systems Administrator"

  },

  {

    "name": "Eli Jones",

    "username": "ejones",

    "phone": {

      "office": "684-348-1127"

    },

    "department": "IT Infrastructure",

    "role": "IT Specialist"

  },

]

-----------------------------------------------------

YAML (Yet Another Markup Language)

            has a lot in common with JSON.

 we’re using the yaml.safe_dump() method to serialize our object into YAML:  

import yaml

with open('people.yaml', 'w') as people_yaml:

    yaml.safe_dump(people, people_yaml)

That code will generate a people.yaml file that looks like this:  

- department: IT Infrastructure

  name: Sabrina Green

  phone:

    cell: 802-867-5310

    office: 802-867-5309

  role: Systems Administrator

  username: sgreen

- department: IT Infrastructure

  name: Eli Jones

  phone:

    office: 684-348-1127

  role: IT Specialist

  username: ejones

JSON is used frequently for transmitting data between web services, while YAML is used the most for storing configuration values.

We've left out some other pretty common ones like

            Python pickle,

             Protocol Buffers, or the             eXtensible Markup Language (XML).

Module 2

More About JSON

JSON (JavaScript Object Notation), which you'll be using in the lab at the end of this module.

JSON is human-readable,

Lots of web services send messages back and forth using JSON.

JSON supports very basic data types

Supports the following objects and types by default:

Python	JSON
dict	object
list, tuple	array
str	string
int, float, int- & float-derived Enums	number
True	true
False	false
None	null

Changed in version 3.4: Added support for int- and float-derived Enum classes.

JSON has

                    strings,     like "Saeed Hassan"

                    Numbers, like 123

                    Objects,     like

{

  "name": "Sabrina Green",

  "username": "sgreen",

  "uid": 1002

 }

                    And a key-value pair can contain another object as a value.  

{

  "name": "Sabrina Green",

  "username": "sgreen",

  "uid": 1002,

  "phone": {

                        "office": "802-867-5309",

                        "cell": "802-867-5310"

                      }

}

As in this case "phone": value is another object

  "phone": {

                        "office": "802-867-5309",

                        "cell": "802-867-5310"

                      }

JSON has arrays, which are equivalent to Python lists. Arrays can contain strings, numbers, objects, or other arrays.  

[

  "apple",

  "banana",

  12345,

  67890,

  {

    "name": "Sabrina Green",

    "username": "sgreen",

    "phone": {

              "office": "802-867-5309",

              "cell": "802-867-5310"

                    },

    "department": "IT Infrastructure",

    "role": "Systems Administrator"

  }

]

==========================================

The json library will help us turn Python objects into JSON, and turn JSON strings into Python objects! The dump() method serializes basic Python objects, writing them to a file. Like in this example:

import json

people = [

  {

    "name": "Sabrina Green",

    "username": "sgreen",

    "phone": {

      "office": "802-867-5309",

      "cell": "802-867-5310"

    },

    "department": "IT Infrastructure",

    "role": "Systems Administrator"

  },

  {

    "name": "Eli Jones",

    "username": "ejones",

    "phone": {

      "office": "684-348-1127"

    },

    "department": "IT Infrastructure",

    "role": "IT Specialist"

  }

]

with open('people.json', 'w') as people_json:

    json.dump(people, people_json)  

================================================

JSON doesn't need to contain multiple lines, but it sure can be hard to read the result if it's formatted this way! Let's use the indent parameter for json.dump() to make it a bit easier to read.  

with open('people.json', 'w') as people_json:

    json.dump(people, people_json, indent=2)

The resulting file should look like this:

[

  {

    "name": "Sabrina Green",

    "username": "sgreen",

    "phone": {

      "office": "802-867-5309",

      "cell": "802-867-5310"

    },

    "department": "IT Infrastructure",

    "role": "Systems Administrator"

  },

  {

    "name": "Eli Jones",

    "username": "ejones",

    "phone": {

      "office": "684-348-1127"

    },

    "department": "IT Infrastructure",

    "role": "IT Specialist"

  }

]

---------------------------------------------------------------------------------------------

>>> pj=json.dumps(peoples, indent=2) >>> pj

'[\n {\n "name": "Sabrina Green",\n "username": "sgreen",\n "phone": {\n "office": "802-867-5309",\n "cell": "802-867-5310"\n },\n "department": "IT Infrastructure",\n "role": "Systems Administrator"\n },\n {\n "name": "Eli Jones",\n "username": "ejones",\n "phone": {\n "office": "684-348-1127"\n },\n "department": "IT Infrastructure",\n "role": "IT Specialist"\n }\n]'

---------------------------------------------------------------------------------

>>> print(pj) [ { "name": "Sabrina Green", "username": "sgreen", "phone": { "office": "802-867-5309", "cell": "802-867-5310" }, "department": "IT Infrastructure", "role": "Systems Administrator" }, { "name": "Eli Jones", "username": "ejones", "phone": { "office": "684-348-1127" }, "department": "IT Infrastructure", "role": "IT Specialist" } ]

----------------------------------------

The load() method does the inverse of the dump() method. It deserializes JSON from a file into basic Python objects. The loads() method also deserializes JSON into basic Python objects, but parses a string instead of a file.

>>> import json

>>> with open('people.json', 'r') as people_json:

...     people = json.load(people_json)

... 

>>> print(people)

[{'name': 'Sabrina Green', 'username': 'sgreen', 'phone': {'office': '802-867-5309', 'cell': '802-867-5310'}, 'department': 'IT Infrastructure', 'role': 'Systems Administrator'}, {'name': 'Eli Jones', 'username': 'ejones', 'phone': {'office': '684-348-1127'}, 'department': 'IT Infrastructure', 'role': 'IT Specialist'}, {'name': 'Melody Daniels', 'username': 'mdaniels', 'phone': {'cell': '846-687-7436'}, 'department': 'User Experience Research', 'role': 'Programmer'}, {'name': 'Charlie Rivera', 'username': 'riverac', 'phone': {'office': '698-746-3357'}, 'department': 'Development', 'role': 'Web Developer'}]

Module 2 The Python Requests Library

The Python Requests library makes it super easy to write programs that send and receive HTTP

>>> import requests

>>> response = requests.get('https://www.google.com') 

>>> print(response.text[:300])

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="de"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title><script nonce="dZfbIAn803LDGXS9

>>> response = requests.get('https://www.google.com', stream=True)

>>> print(response.raw.read()[:100])

b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x02\xff\xc5Z\xdbz\x9b\xc8\x96\xbe\xcfS`\xf2\xb5-\xc6X\x02$t\xc28\xe3v\xdc\xdd\xee\xce\xa9\xb7\xdd;\xe9\x9d\xce\xf6W@\t\x88\x11`@>D\xd6\x9b\xce\xe5<\xc3\\\xcd\xc5\xfc\xab8\x08\xc9Nz\x1f.&\x8e1U\xb5j\xd5:\xfc\xb5jU\x15\x87;^\xe2\x16\xf7)\x97\x82b\x1e\x1d\x1d\xd2S'

We can check out the headers stored in our object to see that the Requests module told the web server that it was okay to compress the content:  

>>> response.request.headers['Accept-Encoding']

'gzip, deflate'

And then the server told us that the content had actually been compressed.  

>>> response.headers['Content-Encoding']

'gzip'

Module 2

Useful Operations for Python Requests

There's a ton of things that we can do with Python Requests.

know if a request we made got a successful response?

>>> response.ok

True

------------------------------------------------

>>> response.status_code

200

---------------------------------------

response = requests.get(url)

if not response.ok:

    raise Exception("GET failed with status code {}".format(response.status_code))

-----------------------------------------------------------------------------

Module 2

HTTP GET and POST Methods

The HTTP GET method, of course, retrieves or gets the resource specified in the URL.

By sending a GET request to the web server, you’re asking for the server to GET the resource for you.

>>> p = {"search": "grey kitten",

...      "max_results": 15}

>>> response = requests.get("https://example.com/path/to/api", params=p)

>>> response.request.url

'https://example.com/path/to/api?search=grey+kitten&max_results=15'

-----------------------------------------------------------------------------

using the HTTP POST method. This method sends, or posts, data to a web service. Whenever you fill a web form and press a button to submit, you're using the POST method to send that data back to the web server.

>>> p = {"description": "white kitten",

...      "name": "Snowball",

...      "age_months": 6}

>>> response = requests.post("https://example.com/path/to/api", data=p)

-----------------------------------------------------------------------------

>>> response.request.url

'https://example.com/path/to/api'

-----------------------------------------------------------------------------

>>> response.request.body

'description=white+kitten&name=Snowball&age_months=6'

-----------------------------------------------------------------------------

Today, it's super common to send and receive data specifically in JSON format, so the Requests module can do the conversion directly for us, using the json parameter.  

>>> response = requests.post("https://example.com/path/to/api", json=p)

>>> response.request.url

'https://example.com/path/to/api'

>>> response.request.body

b'{"description": "white kitten", "name": "Snowball", "age_months": 6}' 

If you want to learn more, feel free to work through the Requests Quickstart.

Module 2

What is Django?

very simple web application created using Django. Django is a full-stack web framework written in Python

Basic components of Web frameworks

     Web frameworks are commonly split into three basic components:

        (1) the application code, where you'll add all of your             application's logic;

        (2) the data storage, where you'll configure what data you want to store and how you're storing it; and

        (3) the web server, where you'll state which pages are served by which logic.

Django makes it easy to interact with data stored in a database by using an object-relational mapper, or ORM

Alternative Python-based web frameworks similar to Django include Flask, Bottle, CherryPy, and CubicWeb.

Module 2

Project Problem Statement

in charge of storing and displaying the customer reviews of the company.

    1. The reviews are stored in text files in the local disk.

    2. Your script should open those files,

    3. process the information to turn it into the format expected by the         web service, then (JSON)

    4. send it to the web service to get stored.

Module 2

Glossary terms from course 6, module 2

API endpoint:

        The part of the program that listens on the network for API calls

Data serialization (C onversion):

         The process of taking an in-memory data structure, like a         Python object, and turning it into something that can be stored on disk or transmitted across a network

Flask:

        A Python library that makes it easier to create web applications and REST web services

JSON:

        A data-interchange format used in RESTful APIs to facilitate communication between clients and servers

REST (Representational State Transfer):

        Every request carries all the parameters and data needed for the server to satisfy that request

RESTful APIs:

        Rely on the HTTP protocol, can be further secured using HTTPS, and API endpoints can authenticate users via authorization tokens, API keys, or other security mechanisms

REST architecture:

        An architectural style for designing networked applications and web services

Richardson Maturity Model (RMM):

        A framework that categorizes and describes different levels of implementation for RESTful APIs based on their adherence to the six constraints

Web application:

        An application that you interact with over HTTP

Module 2

Process text files with Python dictionaries and upload to running web service

Graded Quiz

Process text files with Python dictionaries and upload to running web service

Graded Quiz. • 50 min

DueApr 9, 12:00 AM PKT

Congratulations! You passed!

Grade received 100%

Latest Submission Grade 100%

To pass 80% or higher

This graded quiz assesses your understanding of the concepts and procedures covered in the lab you just completed. Please answer the questions based on the activities you performed in the lab.

Note:

• You can refer to your completed lab for help with the quiz.

• In order to complete this quiz, you must have completed the lab before it.

1.

Question 1

Which of the following methods is used to turn data into Python dictionaries and then pass it as the data attribute of a POST request in Python?

1 / 1 point

requests.post()

dict()

pickle.dumps()

json.loads()

Correct

2.

Question 2

In JSON, which of the following statements accurately describes the format of a key-value pair?

1 / 1 point

Keys and values should be enclosed in parentheses.

Keys and values must be enclosed in single quotes.

Keys and values must be enclosed in double quotes.

Values can only be numbers or strings.

Correct

3.

Question 3

What is the primary purpose of a web framework in software development?

1 / 1 point

To provide a pre-built structure and components for building web applications.

To optimize web browsers for faster loading times.

To design graphic user interfaces for web applications.

To secure web servers against cyber-attacks.

Correct

4.

Question 4

What is the primary purpose of Python's requests library?

1 / 1 point

It is used for database interactions

It is used for creating GUI applications

It is used for scientific computing

It is used for handling HTTP requests

Correct

5.

Question 5

Which of the following is the required parameter for the os.listdir() method?

1 / 1 point

No required parameters

A path to the directory whose contents you want to list

A file descriptor

A list of file extensions

Correct

6.

Question 6

Which of the following statements best describes one of the constraints of REST architecture?

1 / 1 point

REST APIs can maintain client sessions, allowing stateful interactions between the client and server.

REST allows clients to request a specific version of the API, ensuring backward compatibility.

REST requires that the server does not store any client-specific session information between requests, making it stateless.

REST discourages the use of caching to minimize response time and improve server performance.

Correct

7.

Question 7

What is the primary purpose of the HTTP GET method in web development?

1 / 1 point

To update data on a server

To delete data from a server

To request data from a server

To submit data to a server for processing

Correct

8.

Question 8

What is the primary purpose of Django in web development?

1 / 1 point

Creating interactive and responsive web interfaces

Building web applications using JavaScript

Designing database schemas for mobile applications

Developing web applications using the Python programming language

Correct

9.

Question 9

Which of the following is a common use case for the cat command?

1 / 1 point

Creating a new script

Executing code

Displaying a file’s contents

Compiling modules

Correct

10.

Question 10

In the lab, you used the status_code attribute. What does a 200 status code typically indicate?

1 / 1 point

Success - OK

Server error - Internal Server Error

Redirection - Moved Permanently

Client error - Bad Request

Correct

Module 3 Module 3 Introduction

You will lean:

        automating the generation of nicely formatted output from our scripts, like sending emails.

You'll even learn how to generate PDF files to attach to those emails.

          
        We'll show examples of how you can do a bunch of different operations, like

        1. creating the contents of the email or

        2 the PDF,

        3 attaching a file to an email, and

        4. even sending the email to an SMTP server.

Module 3

Logging

print() can be used for debugging or quick information display.

Built-in logging levels 

Filtering problems by severity level which are Highest sever is in last

            DEBUG,

            INFO,

            WARNING, ERROR, and CRITICAL.

print(open(“my_file.txt”,”r”).read(),file=open(“new_file.txt”,”w”))

Handlers

            Handlers define the output destinations for log messages, allowing you to control where the log data goes, such as

        1. writing to files,

        2. sending emails, or

        3. printing to the console. 

The most basic handler is StreamHandler. The StreamHandler class allows you to configure logging to display log messages on the screen like print()

Basic Logging Tutorial¶

https://docs.python.org/3/howto/logging.html#logging-basic-tutorial

Key takeaways

• Overhead of print()statements in production:  

• Distinguishing severity levels for effective monitoring:

• Using logging handlers for advanced log management:

Module 3

The logging module

keep a log of their activities to aid in troubleshooting problems or monitoring the application’s performance. 

The Python logging module provides a standardized way of outputting those log messages to 

            1. the system console, 

            2. a file, 

            3. system logging daemon (syslogd), or 

            4. other  destinations.

Five levels of the logging module

The severity levels are as follows:

• DEBUG:

• Extra information only needed for debugging. This level often includes values of parameters and variables to help with troubleshooting. Example: logging Parameter x: 42 to aid in diagnosing issues and include parameter values

• INFO:

• Informational message that can be used for tracing the program’s activities. Example: connecting to MySQL database at mydbserver.aws.amazon.com

• WARNING:

• An error that doesn’t require immediate action. Examples: query took longer than 10 seconds, configuration file not found, or reverting to default settings.

• ERROR:

• Serious, but can be a recoverable error. Example: database connection failed, or a file is missing.

• CRITICAL:

• Fatal error. Example: A critical error has occurred! The application will now terminate.

A simple way to initialize the logger is the following

# Set the minimum logging level to INFO,

logging.basicConfig(level=logging.INFO)

# Get a logger object

log = logging.getLogger(__name__)

# Start the log file

log.info("Hello world")

Alternatives to the logging module 

One popular alternative is structlog, which outputs logs in a highly structured JSON format so they can be easily parsed by monitoring systems

Another is colorlog, which adds helpful color highlights when logging to the console. An example of this is assigning different colors to log levels like DEBUG, INFO, WARNING, ERROR, and CRITICAL.  

Module 3 Exception handling                  

Some examples of exceptions are the following:

1. Attempting to divide by zero
2. Referencing a variable that doesn’t exist
3. Attempting to open a file that doesn’t exist
4. Connecting fails to a remote server

The most common types of exceptions

• NameError - usually due to a typo in a variable name

• AttributeError - also usually a typo, in calling a method on an object

• ValueError - parameter value is incorrect

• TypeError - sending a string when a function is expecting an int or calling a function with the wrong number or type of arguments

• ImportError - when Python can’t find a module you’re trying to import

• FileNotFoundError - when you try to perform file-related operations (opening, reading, writing, or deleting) on a file or directory that does not exist

An example of an except statement is:

 try:

  # Try to append to a file that is normally not writable

  # for anyone other than root 

  f = open("/etc/hosts", "w+")

except IOError as ex:

  # The variable "ex" will hold details about the error

  # that occurred

  print("Error appending to file: " + str(ex))

else:

  # If there was no exception, close the file.

  f.close() 

   ------------------------------------------------------------------------------

 x = "hello"

if not isinstance(x, int):

  raise TypeError("Only integers are allowed")

 ------------------------------------------------------------------------------

 def start_server(port):

  if not isinstance(port, int):

    raise TypeError("Port number must be an integer")

  elif port < 1024 or port > 65535:

    raise ValueError("Port number is invalid")

start_server(1024)

 ------------------------------------------------------------------------------

try:

  f = open("/etc/hosts", "w+")

except:

  print("Error appending to file: " + str(ex))

finally:

  f.close()  # causes error if the file could not be opened

 ------------------------------------------------------------------------------

Key takeaways

        These commands include try/except, raise, and finally. The try/except exception allows you to catch errors before they happen.

        The finally exception helps you clean up after executing a code, whether that code raises an exception or not.

  ------------------------------------------------------------------------------

Module 3 Introduction to Python Email Library

 

The

Simple Mail Transfer Protocol (SMTP) and Multipurpose Internet Mail Extensions (MIME)

. The email built-in Python module lets us easily construct email messages.

 ------------------------------------------------------------------------------

from email.message import EmailMessage

message = EmailMessage()

print(message)

# no any output

# it is only empty Dictionary

sender = "me@example.com"

recipient = "you@example.com"

message['From'] = sender

message['To'] = recipient

print(message)

From: me@example.com

To: you@example.com

message['Subject'] = 'Greetings from {} to {}!'.format(sender, recipient)

print(message)

From: me@example.com

To: you@example.com

Subject: Greetings from me@example.com to you@example.com!

body = """Hey there!

I'm learning to send emails using Python!"""

message.set_content(body)

print(message)

From: me@example.com

To: you@example.com

Subject: Greetings from me@example.com to you@example.com!

MIME-Version: 1.0

Content-Type: text/plain; charset="utf-8"

Content-Transfer-Encoding: 7bit

Hey there!

I'm learning to send email using Python! 

 ------------------------------------------------------------------------------

Uptill now we only made dictionary with name message with keys are

              From 

              To

              Subject

              Body

In the preceding section we will send email using protocol MIME

 

 

 Module 3

Introduction to Generating PDFs

 

Python PDF generated tool 

ReportLab https://docs.reportlab.com/

 fruit = {

  "elderberries": 1,

  "figs": 1,

  "apples": 2,

  "durians": 3,

  "bananas": 5,

  "cherries": 8,

  "grapes": 13

}

 -----------------------------------------------------------------------------------

>>> from reportlab.platypus import SimpleDocTemplate

>>> report = SimpleDocTemplate("d:\\report.pdf") 

--------------------------------------------------------------------------------- 

 >>> from reportlab.platypus import Paragraph, Spacer, Table, Image

 --------------------------------------------------------------------------------- 

>>> from reportlab.lib.styles import getSampleStyleSheet

>>> styles = getSampleStyleSheet()

  --------------------------------------------------------------------------------- 

>>> report_title = Paragraph("A Complete Inventory of My Fruit", styles["h1"])

  --------------------------------------------------------------------------------- 

>>> report.build([report_title])

 

 Module 3

Adding Tables to our PDFs

 

Let's spice this up by adding a Table. To make a Table object, we need our data to be in a list-of-lists, sometimes called a two-dimensional array. We have our inventory of fruit in a dictionary. How can we convert a dictionary into a list-of-lists?

>>> table_data = []

>>> for k, v in fruit.items():

...   table_data.append([k, v])

...

>>> print(table_data)

[['elderberries', 1], ['figs', 1], ['apples', 2], ['durians', 3], ['bananas', 5], ['cherries', 8], ['grapes', 13]]

--------------------------------------------------------------------------------- 

>>> report_table = Table(data=table_data)

>>> report.build([report_title, report_table])

---------------------------------------------------------------------------------

 

 

 

 

 

  

 

 

 TableStyle definitions can get pretty complicated,

 ---------------------------------------------------------------------------------

 >>> from reportlab.lib import colors

 

>>> table_style = [('GRID', (0,0), (-1,-1), 1, colors.black)]

 

>>> report_table = Table(data=table_data, style=table_style, hAlign="LEFT")

 

>>> report.build([report_title, report_table])

---------------------------------------------------------------------------------

 

 

 

 

 

 

 

Module 3

Adding Graphics to our PDFs

We’re going to need to use the Drawing Flowable class to create a Pie chart. 

---------------------------------------------------------------------------------

from reportlab.graphics.shapes import Drawing

from reportlab.graphics.charts.piecharts import Pie

report_pie = Pie(width=3*inch, height=3*inch)

---------------------------------------------------------------------------------

To add data to our Pie chart, we need two separate lists: 

                1. One for data, and 

                2. one for labels.

-------------------------------------------------------------------------------

report_pie.data = []

report_pie.labels = []

for fruit_name in sorted(fruit):

    report_pie.data.append(fruit[fruit_name])

    report_pie.labels.append(fruit_name)

print(report_pie.data)

# output: [2, 5, 8, 3, 1, 1, 13]

print(report_pie.labels)

# output: ['apples', 'bananas', 'cherries', 'durians', 'elderberries', 'figs', 'grapes']

-------------------------------------------------------------------------------

The Pie object isn’t Flowable, but it can be placed inside of a Flowable Drawing.

report_chart = Drawing()

report_chart.add(report_pie)

------------------------------------------------------------------------------

report.build([report_title,report_table,report_chart])

-------------------------------------------------------------------------------

 

 You'll want to refer to the ReportLab User Guide

for more details

 

Module 3

Project Problem Statement

 

you'll have to process information related to the sales your company generated last month, and 

turn that into a nicely formatted PDF report that you'll then 

send by email so that your boss can look at it. 

 

Automatically Generate a PDF and sending it by E-mail

 $ cat ~/scripts/reports.py

#!/usr/bin/env python3

import reportlab

from reportlab.platypus import SimpleDocTemplate
from reportlab.platypus import Paragraph, Spacer, Table, Image
from reportlab.lib.styles import getSampleStyleSheet
from reportlab.lib import colors

def generate(filename, title, additional_info, table_data):
  styles = getSampleStyleSheet()
  report = SimpleDocTemplate(filename)
  report_title = Paragraph(title, styles["h1"])
  report_info = Paragraph(additional_info, styles["BodyText"])
  table_style = [('GRID', (0,0), (-1,-1), 1, colors.black),
                ('FONTNAME', (0,0), (-1,0), 'Helvetica-Bold'),
                ('ALIGN', (0,0), (-1,-1), 'CENTER')]
  report_table = Table(data=table_data, style=table_style, hAlign="LEFT")
  empty_line = Spacer(1,20)
  report.build([report_title, empty_line, report_info, empty_line, report_table])

$ cat ~/scripts/emails.py

#!/usr/bin/env python3

import email.message
import mimetypes
import os.path
import smtplib

def generate(sender, recipient, subject, body, attachment_path):
  """Creates an email with an attachement."""
  # Basic Email formatting
  message = email.message.EmailMessage()
  message["From"] = sender
  message["To"] = recipient
  message["Subject"] = subject
  message.set_content(body)

  # Process the attachment and add it to the email
  attachment_filename = os.path.basename(attachment_path)
  mime_type, _ = mimetypes.guess_type(attachment_path)
  mime_type, mime_subtype = mime_type.split('/', 1)

  with open(attachment_path, 'rb') as ap:
    message.add_attachment(ap.read(),
                          maintype=mime_type,
                          subtype=mime_subtype,
                          filename=attachment_filename)

  return message

def send(message):
  """Sends the message to the configured SMTP server."""
  mail_server = smtplib.SMTP('localhost')
  mail_server.send_message(message)
  mail_server.quit()

$ ./scripts/example.py

Here, you'll need a login to roundcube using the student as username and password for the password, followed by clicking Login.

$ nano ~/scripts/example.py

#!/usr/bin/env python3


import emails
import os
import reports

table_data=[['Name', 'Amount', 'Value'],
           ['elderberries', 10, 0.45],
           ['figs', 5, 3],
           ['apples', 4, 2.75],
           ['durians', 1, 25],
           ['bananas', 5, 1.99],
           ['cherries', 23, 5.80],
           ['grapes', 13, 2.48],
           ['kiwi', 4, 0.49]]
reports.generate("/tmp/report.pdf", "A Complete Inventory of My Fruit", "This is all my fruit.", table_data)
sender = "automation@example.com"
receiver = "{}@example.com".format(os.environ.get('USER'))
subject = "List of Fruits"
body = "Hi\n\nI'm sending an attachment with all my fruit."
message = emails.generate(sender, receiver, subject, body, "/tmp/report.pdf")
emails.send(message)

$ cat ~/car_sales.json

[{"id":1,"car":{"car_make":"Ford","car_model":"Club Wagon","car_year":1997},"price":"$5179.39","total_sales":446},
{"id":2,"car":{"car_make":"Acura","car_model":"TL","car_year":2005},"price":"$14558.19","total_sales":589},
{"id":3,"car":{"car_make":"Volkswagen","car_model":"Jetta","car_year":2009},"price":"$14879.11","total_sales":825},
{"id":4,"car":{"car_make":"Chevrolet","car_model":"Uplander","car_year":2006},"price":"$17045.06","total_sales":689},
{"id":5,"car":{"car_make":"Plymouth","car_model":"Roadrunner","car_year":1969},"price":"$14770.44","total_sales":691},
{"id":6,"car":{"car_make":"GMC","car_model":"Safari","car_year":2000},"price":"$13390.83","total_sales":531},
{"id":7,"car":{"car_make":"Lamborghini","car_model":"Murciélago","car_year":2003},"price":"$7267.94","total_sales":374},
{"id":8,"car":{"car_make":"GMC","car_model":"3500","car_year":1999},"price":"$19292.10","total_sales":638},
{"id":9,"car":{"car_make":"Maybach","car_model":"62","car_year":2004},"price":"$11020.45","total_sales":945},
{"id":10,"car":{"car_make":"Chevrolet","car_model":"Cavalier","car_year":2001},"price":"$10708.87","total_sales":870},

{
        "id": 47,
        "car": {
                "car_make": "Lamborghini",
                "car_model": "Murciélago",
                "car_year": 2002
        },
        "price": "$13724.05",
        "total_sales": 149
}

nano ~/scripts/cars.py

,car["car_year"])

max_revenue["car"]), max_revenue["revenue"]),

why locale.atof is used why not simple use float()

price"], item["total_sales"]])

Automatically generate a PDF and send it by email

Graded Quiz

 

 

DueApr 16, 12:00 AM PKT

Congratulations! You passed!

Grade received 87.50%

Latest Submission Grade 87.50%

To pass 80% or higher

Retake the assignment in 23h 42m

This graded quiz assesses your understanding of the concepts and procedures covered in the lab you just completed. Please answer the questions based on the activities you performed in the lab.

Note:

• You can refer to your completed lab for help with the quiz.

• In order to complete this quiz, you must have completed the lab before it.

1.

Question 1

The emails.py script defines the generate function. Which commands in the example.py script make it possible to call this function? Select all that apply.

1 / 1 point

import os

import reports

Correct

reports.generate

Correct

emails.send

2.

Question 2

What step must you complete before you can run the example.py script?

1 / 1 point

ls ~/scripts

./scripts/example.py

cat ~/scripts/example.py

sudo chmod o+wx ~/scripts/example.py

Correct

3.

Question 3

You attempt to run the script example.py but get the error message “Permission denied”. What should you do?

1 / 1 point

Use the nano command to edit the script.

Use the cat command to view the file and look for the error.

Run example.py in debug mode.

Use the sudo chmod command to enable you to execute the file.

Correct

4.

Question 4

You’re ready to add Python code to calculate the car with the most sales. What command must you use to open the file for editing?

1 / 1 point

nano ~/scripts/cars.py

ls ~/scripts/cars.py

cat ~/scripts/cars.py

chmod ~/scripts/cars.py

Correct

5.

Question 5

You’ve edited the cars.py script to generate the cars.pdf but find that all the information in the PDF is on a single line. What should you do to fix the report?

1 / 1 point

Edit the cars_dict_to_table function to include <br/> between the lines.

Edit the reports.generate function to include <br/> between the lines.

Edit the process_data function to include <br/> between the lines.

Edit the format_car method to include <br/> between the lines.

Correct

6.

Question 6

What is the primary purpose of the example.py script?

1 / 1 point

Managing user authentication for a website

Analyzing data from a database

Generating PDF reports and sending them via email

Generating and printing text documents

Correct

7.

Question 7

What library enables you to send emails from a Python script?

1 / 1 point

smtplib

os.path

email.message

mimetypes

Correct

8.

Question 8

What is the purpose of the following commands from the reports.py script? Select all that apply.

1

2

import reportlab

from reportlab.platypus import SimpleDocTemplate

0.75 / 1 point

To specify the format of the PDF

Correct

To create the PDF

This should not be selected

Review Qwiklabs assessment: Automatically generate a PDF and send it by email.

To specify the format of the email message

To import the reportlab library

Correct

9.

Question 9

What is the purpose of the if item_revenue > max_revenue[“revenue”]: line in the cars.py script?

0 / 1 point

To exit the function when a car’s revenue is higher than max revenue

To determine whether a car should be included in the output table

To find out which car generated the lowest sales

To identify which car generated the highest sales

Incorrect

Review Qwiklabs assessment: Automatically generate a PDF and send it by email.

10.

Question 10

You’ve completed the task in the lab to send an email with the sales summary car information. However, the body of the email has one long paragraph. What should you do to improve formatting in the body of the email?

1 / 1 point

Put the body of the email in the PDF

Include empty_line = Spacer (1,20) to insert line breaks

Include \n to insert line breaks

Include <br/> to insert line breaks

Correct

 

Some attempts' wrong Answers

Question 1

What is the overall purpose of the reports.py script in the lab?

0 / 1 point

To send a report

To create a PDF of the report

To format a report

To pint a report

Incorrect

 

Question 5

In the cars.py script, which method must you edit to calculate the car model with the most sales?

1 / 1 point

cars_dict_to_table

load_data

process_data

format_car

Correct

 

Question 6

In the example.py script, what is the purpose of the three import commands?

0 / 1 point

To retrieve another script and execute it before the example.py script runs.

To retrieve the data needed for the PDF.

To convert the data into the correct format for the PDF.

To enable a script to use commands from an imported module.

Incorrect

Question 8

What is the purpose of the following commands from the reports.py script? Select all that apply.

from reportlab.platypus import SimpleDocTemplate

import reportlab

0.5 / 1 point

To specify the format of the email message

To specify the format of the PDF

To import the reportlab library

Correct

To create the PDF

This should not be selected

Question 9

In the cars.py script, what is the purpose of the format_car function?

0 / 1 point

To display the car make and price

To display the car model and car year

To display the car ID, car make, car model, and car year information

To display the car make, car model, and car year

Incorrect

 

Which command identifies the mail host that will send the email generated by your script?

0 / 1 point

mail_server.send_message(message)

message = email.message.EmailMessage()

import smtplib

mail_server = smtplib.SMTP(‘localhost’)

Incorrect

 

Question 10

Which commands must be included in your script if you want to send an email? Select all that apply.

0.75 / 1 point

emails.send

Correct

emails.generate

import emails

Correct

reports.generate

You didn’t select all the correct answers

 

Module 3 DevOps

Five DevOps principles

CCollaboration:

The main purpose of DevOps is to promote and facilitate collaboration among teams. (Development and Operation )

2.  Automation:

Each part of the development process is automated. This helps make the building, testing, deploying, and monitoring of the code more stable and predictable.

3.  Continuous improvement:

Teams should always be looking for opportunities to improve the process, tools, and communication between teams.

4.  Customer mindset:

Teams should remember that the software they are developing will be used by customers. Listen to customers and take their feedback into consideration to continually create an improved software program. You want to develop software that is fun to use.

5.  Create with the end in mind:

Teams should understand customer needs and pain points, and they should develop solutions that solve real problems.

Key takeaways

The DevOps philosophy is an opportunity for the

1.   development

2.   operations teams

to work together in an agile environment and optimize

1.   the development and

2.   deployment process.

For DevOps to be successful, team members need to be open to working in a collaborative way with new processes.

The automation tools can take time to learn how to implement,

Module 3 SLAs

Service-Level Agreements

You can think of these as promises you are giving your users. And as with any promise there are consequences for breaking that promise.

An SLO (service level objective) is an agreement within an SLA

 about a specific metric like uptime or response time. 

 

How SLAs relate to SLOs

 

SLA examples

Service-level agreement (SLA) examples and template  

 

 

Key takeaways 

        An SLA is a service agreement typically between an IT service provider and a client. It outlines the details of the service, including

            1. what the service should accomplish and

            2. the consequences if there is a breach in the contract. 

Module 3  SLOs

Service-level objectives     

    

SLOs are found within an SLA and focus on specific and measurable metrics like uptime and response time.

SLOs set the customers’ expectations and communicate to the IT and DevOps teams the goals they need to achieve  

    

Key takeaways

SLOs are important as they help achieve promises defined in the SLA.

These help set customer expectations and should be clearly written, measurable, and attainable. 

 

 Module 3  SLIs

service-level indicator

SLIs measure the performance of an application at any given time.

It’s important to monitor SLIs to determine if your application is meeting the objective — or violating the SLA.

Key takeaways 

Service-level indicators help make service-learning objectives measurable. They provide actual data on performance.

Module 3

Error budgets

 

An error budget is the maximum amount of time a software program can fail and still be in compliance with the service-level objective (SLO).

An error budget is typically represented by a percentage.

A simple example is this: If an SLO states that a website should function properly 99.9% of the time, then the error budget is only 0.1%. 

Module 3

DevOps review

Automation is a key element through the entire DevOps process.

DevOps has five key principles that include:

1.   collaboration,

2.   automation,

3.   continuous improvement,

4.   ustomer mindset, and

5.   end-goal-focused creation.

 

A service-level agreement (SLA)

Service-level objectives (SLOs)

Service-level indicators (SLIs

 

An error budget is the maximum amount of time a software program can fail and still be in compliance with the SLO. 

 

Module 3

Practice quiz: DevOps 

 

Congratulations! You passed!

Grade received 80%

To pass 80% or higher

1.

Question 1

You work for a start-up tech company and your client is asking questions like, “What will happen if the system goes down?” and, “How fast will you be able to fix it?” You tell the client that the answer to these questions will be found in the contract. What are you referring to?

1 / 1 point

SLO

SLI

SLA

DevOps

Correct

2.

Question 2

You’ve been asked to provide new software developers with insights on service-level objectives. What is the purpose of a service-level objective?

1 / 1 point

To define measurable and acceptable levels of performance for a software service

To identify the physical location of the server infrastructure

To differentiate the pricing tiers for different services

To identify the maximum number of users a server can have

Correct

3.

Question 3

You’ve been asked to craft service-level indicators (SLIs) for a new software service. What is the best way to describe an SLI?

1 / 1 point

A language used to develop microservices

A metric that focuses on the quality of customer support

A measurement that tracks the performance of a service

A metric that focuses on tier pricing

Correct

4.

Question 4

You’ve been asked to speak at the next software developer conference about DevOps. What is the role of automation in DevOps?

0 / 1 point

To eliminate the need for human intervention using automation tools

To streamline and enhance efficiency in development and operations teams

To create a more complex workflow

To increase the amount of manual processes

Incorrect

Not quite. Automation can help eliminate some need for human intervention, it doesn’t eliminate it entirely.

5.

Question 5

Your manager asked you what the updated error budget is for this month’s software program. What do they mean by error budget?

1 / 1 point

The maximum number of errors a software program can have at launch time

The maximum budget allowed for hardware maintenance

The maximum amount of time a developer has to correct code errors during development

The maximum amount of time for which a software program can fail and still be in compliance with the service-level objective

Correct

 Module 3

Glossary terms from course 6, module 3

Error budgets:

Represented as the maximum amount of time that a program is able to fail without violating an agreement

Service-level agreements (SLAs):

An agreement between a vendor and its clients or users that can be legally binding

Service-level objectives (SLOs):  

A specific and measurable target that defines the level of performance, reliability, or quality a service should consistently deliver

 Module 3

IT skills in action reading

Migrating data 

We have to migrate data from old legacy system to modern system. The manual migration would be very costly so. So Adam decided to migrate data automatically using Python APIs.

The affordable solution

Adam decided to use his Python programming skills to migrate the data instead. He called APIs and the requests module to extract the data from SupportGenius Solutions (Old legacy) and processed everything on his own. Then, he loaded the data onto SupportWave Technologies (Modern Sophisticated System).

One downside to SupportWave Technologies that Adam discovered was that its API was prone to random errors.

Adam decided to use Python’s exception handling mechanism to retry operations until they were completed.

Adam used his Python skills to not only migrate data from one customer support system to another, and he also did it affordably, allowing him to handle the growing customer service inquiries he was receiving.

 Module 4

Module 4 Introduction

 

In the past few modules, you've seen

1.   how you can modify images using Python Imaging Library;

2.   how you can interact with web services using the Python requests module,

3.   sending data in JSON format;

4.   how you can generate PDF files with the content you want; and how you can send emails with those PDFs as an attachment. 

For the final project in this course, you’ll use the techniques and concepts you've seen to build a solution to a complex IT task.

This can seem a bit daunting at first, but don't worry, you already have all the tools to solve this task!

Module 4

Project Problem Statement

 

Okay, here's the scenario:

You work for an online fruit store, and you need to develop a system that will update the catalog information with data provided by your suppliers. When each supplier has new products for your store, they give you an image and a description of each product.

Given a bunch of images and descriptions of each of the new products, you’ll:

·         Upload the new products to your online store. Images and descriptions should be uploaded separately, using two different web endpoints.

·         Send a report back to the supplier, letting them know what you imported.

Since this process is key to your business's success, you need to make sure that it keeps running! So, you’ll also:

·         Run a script on your web server to monitor system health.

·         Send an email with an alert if the server is ever unhealthy.

Hopefully this summary has helped you start thinking about how you’ll approach this task. In case you’re feeling a little scared, don't worry, you can definitely do this! You have all the necessary tools, and the lab description will go into a lot more detail of what you need to do. 

 Module 4

How to Approach the Problem

Break the problem down into smaller pieces.

Make one change at a time.

        Do the unit test on the change you made

Use version control.

        So that you can rollback the program to its good state.

Review module documentation! 

feel free to go back and review them if you need a refresher!

• Python Image Library (PIL) - Tutorial

• Requests (HTTP client library) - Quickstart

• ReportLab (PDF creation library)

• email (constructing email)

• psutil (processes and system utilization)

• shutil (file operations)

• smtplib (sending email)

 Module 4 Qwiklabs assessment: Automate updates to catalog information  

 For solving LAB I GOT some HELP  FROM

https://github.com/untgrad/gia_course6_lab4/tree/master 

 

Automate updating catalog information

Graded Quiz. • 50 min

Congratulations! You passed!

Grade received 90%

Latest Submission Grade 90%

To pass 80% or higher 

 

1.

Question 1

In the lab, what is the purpose of the changeImage.py Python script?

1 / 1 point

It generates PDF reports for supplier data.

It uploads images to a web server.

It processes supplier images to change their resolution and format.

It sends error emails for system health checks.

Correct

2.

Question 2

What is the purpose of using the Pillow (Python Imaging Library) in the changeImage.py script?

1 / 1 point

To process and modify the supplier images

To download images from the internet

To generate PDF reports

To send emails to the supplier

Correct

3.

Question 3

In Python, what method is recommended to convert raw images with alpha transparency layers to RGB images without losing important information?

0 / 1 point

Apply the "rotate" method.

Utilize the "resize" method.

Use the "convert_alpha" method.

Execute the "flip" method.

Incorrect

Review the Qwiklabs assessment: Automate updates to catalog information lab.

4.

Question 4

In the lab, what does changeImage.py primarily aim to do?

1 / 1 point

Check the health status of the system.

Create a PDF report for the processed images.

Process and resize images to specific specifications.

Send images by email to the supplier.

Correct

5.

Question 5

What is the purpose of the report_email.py script in the lab, and what actions does it perform after generating the PDF report?

1 / 1 point

report_email.py generates PDF reports.

report_email.py sends an email to the supplier.

report_email.py uploads images to a web server.

report_email.py processes text data from supplier descriptions.

Correct

6.

Question 6

Which HTTP request method in the requests module is typically used to retrieve data from a web server without making any changes to the server's data?

1 / 1 point

GET

PUT

POST

DELETE

Correct

7.

Question 7

In Python, what step is necessary to make a script runnable as an executable file on a Unix-based system?

1 / 1 point

Wrapping the script in a try...except block

Adding a shebang line at the beginning of the script

Changing the file extension to ".exe"

Using the pip install command to make it executable

Correct

8.

Question 8

What is the purpose of setting up a cron job for the health_check.py script in the given lab?

1 / 1 point

To manually trigger the health_check.py script at specific intervals.

To schedule the script to run automatically at regular intervals and send email alerts if system issues are detected.

To execute the script only once and generate a system health report.

To create a backup of the health_check.py script for future reference.

Correct

9.

Question 9

In the context of setting a user cron job, what is the purpose of running the crontab -e command followed by selecting an editor?

1 / 1 point

To open the user's crontab file for editing.

To view the contents of the user's crontab file.

To delete all existing cron jobs for the user.

To list all existing cron jobs for the user.

Correct

10.

Question 10

What are the key steps involved in the process of updating your company's online website with new products in the lab's scenario?

1 / 1 point

Processing images and descriptions, creating a webmail inbox, and running stress tests.

Creating Python scripts for image and description processing, updating the website, and generating a PDF report.

Setting up a cron job, installing the stress tool, and sending emails with health status.

Generating a PDF report, monitoring system health, and sending emails.

Correct

Module 4

Course 6 glossary

 

 Glossary

IT Automation with Python

Terms and definitions from Course 6

API endpoint: The part of the program that listens on the network for API calls

Data serialization: The process of taking an in-memory data structure, like a Python object,and turning it into something that can be stored on disk or transmitted across a network

Distributed systems: Also referred to as distributed computing or distributed databases,utilize different nodes to interact and synchronize over a shared network

Docstrings: Documentation that lives alongside the code

E

Error budgets: Represented as the maximum amount of time that a program is able to fail

without violating an agreement

F

Flask: A Python library that makes it easier to create web applications and REST web services

J

JSON: A data-interchange format used in RESTful APIs to facilitate communication between

clients and servers

N

NALSD (Non-Abstract Large System Design): A discipline and process introduced by Google, primarily aimed at empowering site reliability engineers (SREs) to assess, design, and evaluate large-scale systems

Naming conventions: Functions, classes and methods with naming conventions to understand what to expect from them

R

REST (Representational State Transfer): Every request carries all the parameters and data

needed for the server to satisfy that request

RESTful APIs: Rely on the HTTP protocol, can be further secured using HTTPS, and API

endpoints can authenticate users via authorization tokens, API keys, or other security

mechanisms

REST architecture: An architectural style for designing networked applications and web

services

Richardson Maturity Model (RMM): A framework that categorizes and describes different

levels of implementation for RESTful APIs based on their adherence to the six constraints

S

Service-level agreements (SLAs): An agreement between a vendor and its clients or users that can be legally binding

Service-level objectives (SLOs): A specific and measurable target that defines the level of performance, reliability, or quality a service should consistently deliver

Web application: An application that you interact with over HTTP