并发编程(Concurrent programming)

并发编程理论

并行

并行是指“并排行走”或“同时实行或实施”。在操作系统中,若干个程序段同时在系统中运行,这些程序的执行在时间上是重叠的,一个程序段的执行尚未结束,另一个程序段的执行已经开始,无论从微观还是宏观,程序都是同时执行的;

并行是指“并排行走”或“同时实行或实施”。在操作系统中,若干个程序段同时在系统中运行,这些程序的执行在时间上是重叠的,一个程序段的执行尚未结束,另一个程序段的执行已经开始,无论从微观还是宏观,程序都是同时执行的;

问:单核CPU能否实现并行?
答:不能,但是可以实现并发

并发

并发是指:在同一个时间段内,两个或多个程序执行,有时间上的重叠(宏观上是同时,微观上仍是顺序执行);
通俗讲,多个程序只要看起来像同时运行即可;

并发是指:在同一个时间段内,两个或多个程序执行,有时间上的重叠(宏观上是同时,微观上仍是顺序执行);

通俗讲,多个程序只要看起来像同时运行即可;

问:12306可以同一时间支持几个亿的用户买票 问是并行还是并发
答:高并发

以上建立在单核CPU上~

以上建立在单核CPU上~

程序和进程

百度百科:进程(Process)是计算机中的程序关于某数据集合上的一次运行活动,是系统进行资源分配和调度的基本单位,是操作系统结构的基础。在早期面向进程设计的计算机结构中,进程是程序的基本执行实体;在当代面向线程设计的计算机结构中,进程是线程的容器。程序是指令、数据及其组织形式的描述,进程是程序的实体。
从开发角度,程序可以理解为一堆代码,而进程是怎么产生的呢?
进程的产生可以理解为:正在运行的程序,是活的,而程序(一堆代码)放在那是死的,不运行也不会产生什么;

百度百科:进程(Process)是计算机中的程序关于某数据集合上的一次运行活动,是系统进行资源分配和调度的基本单位,是操作系统结构的基础。在早期面向进程设计的计算机结构中,进程是程序的基本执行实体;在当代面向线程设计的计算机结构中,进程是线程的容器。程序是指令、数据及其组织形式的描述,进程是程序的实体。

从开发角度,程序可以理解为一堆代码,而进程是怎么产生的呢?

进程的产生可以理解为:正在运行的程序,是活的,而程序(一堆代码)放在那是死的,不运行也不会产生什么;

单核情况下的进程调度

单核情况下的进程调度

1、FCFS:先来先服务,如果先来一个长作业,比如要执行24h的程序,后面只有1s的程序,这样对短作业是不友好的;
2、短作业优先调度算法:相反这样是对长作业不友好;
		    |
    		|
             
3、时间片轮转法+多级反馈队列:先分配给新的多个进程相同的时间片段,之后根据进程消耗时间片多少分类执行;

进程三状态图

就绪、运行、阻塞

就绪、运行、阻塞

并发编程

  • 就绪态:当进程已分配到除CPU以外的所有必要的资源,只要获得处理机便可立即执行,这时的进程状态称为就绪状态。
  • 运行态:当进程已获得处理机,其程序正在处理机上执行,此时的进程状态称为运行状态。
  • 阻塞态:正在执行的进程,由于等待某个事件发生而无法执行时,便放弃处理机而处于阻塞状态。引起进程阻塞的事件可有多种,例如,等待I/O完成、申请缓冲区不能满足、等待信件(信号)等。

程序在运行之后先进入就绪态,直到第一行代码运行的时候进入运行态,如果中间有代码导致程序夯住,就会导致阻塞态;三态会不断切换;

程序在运行之后先进入就绪态,直到第一行代码运行的时候进入运行态,如果中间有代码导致程序夯住,就会导致阻塞态;三态会不断切换;

同步、异步

同步:提交完任务之后原地等待任务的返回结果,期间不做任何事(消耗资源)

异步:提交完任务之后不原地等待任务的返回结果,直接去做其他事 ,结果由反馈机制自动提醒(优化)

# 概念剖析
比如你要做三件事:学习、洗衣服、做饭
# 同步
同步做这三件事的过程:学习期间不能干别的事,不学习了才能去洗衣服,洗衣服期间只能等着洗衣机洗完衣服,不能干别的事,然后衣服洗好了,才能去做饭;
# 结果就是这样又耗时耗力

# 异步
异步做着三件事的过程:衣服丢洗衣机里,米放锅里,然后去学习;这样是不是很轻松,也省时间;

阻塞、非阻塞

二者用于描述任务的执行状态

二者用于描述任务的执行状态

阻塞:是指调用结果返回之前,当前线程会被挂起

比如python中的input方法获取输入,你不输入会一直处于等待状态,也就是阻塞

非阻塞:就是阻塞的反面,执行和运行

同步/异步、阻塞/非阻塞组合

  • 同步阻塞:这种形式效率是最低的;比如你吃饭排队,只能干等,什么都不能干;
  • 异步阻塞:比如你排队吃饭,排到你了让店员喊一声你就好,这期间可以在附近买杯奶茶等(异步),但是不能离开餐厅附件(阻塞);
  • 同步非阻塞:比如吃饭排队,你估计排队时间挺长的,你打开了王者荣耀,你边打游戏还得往前走关注排队情况,这样以来王者荣耀相当于一个程序,排队是一个程序,二者需要不断切换;
  • 异步非阻塞:效率非常高,比如吃饭排队,你可以把排单号交给女朋友,让她排队,你去厕所(异步),拉屎(非阻塞);这样是不是不需要排队也不需要在餐厅旁边等着,这样就是异步非阻塞;

可能栗子不恰当,不要纠结

可能栗子不恰当,不要纠结

并发编程实操

创建进程

在windows中开设进程类似于导入模块,从上往下执行代码,一定需要在判断语句内执行开设进程的代码;
在linux中是直接将代码完整的复制一份执行,不需要在判断语句内执行

在windows中开设进程类似于导入模块,从上往下执行代码,一定需要在判断语句内执行开设进程的代码;

__main__

在linux中是直接将代码完整的复制一份执行,不需要在判断语句内执行

__main__
# 代码层面创建进程
方法一:
from multiprocessing import Process
import time
import os


def test_print(name):
    print(os.getpid())  # 获取进程号
    print(os.getppid())  # 获取父进程号
    print('%s正在运行' % name)
    time.sleep(3)
    print('%s已经结束' % name)


if __name__ == '__main__':
    p = Process(target=test_print, args=('Hammer',))  # 生成一个进程对象
    p.start()  # 告诉操作系统开设一个新的进程,异步提交
    print(os.getpid())
    print('主进程')
    
    
方法二: 采用继承的方法
class MyProcess(Process):
    def __init__(self, name):
        super().__init__()
        self.name = name

    def run(self):
        print('%s正在运行' % self.name)
        time.sleep(3)
        print('%s已经结束' % self.name)

if __name__ == '__main__':
    p = MyProcess('Hammerze')
    p.start()
    print('主进程')

进程的join方法

先进行子进程的代码,执行完子进程执行父进程的代码;
相当于改为串行;

先进行子进程的代码,执行完子进程执行父进程的代码;

相当于改为串行;

from multiprocessing import Process
import time


def test(name, n):
    print('%s is run' % name)
    time.sleep(n)
    print('%s is over' % name)


if __name__ == '__main__':
    p_list = []
    start_time = time.time()
    for i in range(1, 4):
        p = Process(target=test, args=(i, i))
        p.start()
        p_list.append(p)
        p.join()  # 串行  9s+
   #  for p in p_list:     
   #      p.join()
   #  print(time.time() - start_time)

进程间默认无法交互

# 进程间数据是相互隔离的
from multiprocessing import Process

money = 100

def test():
    global money
    money = 999


if __name__ == '__main__':
    p = Process(target=test)
    p.start()
    # 先确保子进程运行完毕了 再打印
    p.join()
    print(money)  # 100,打印的是主进程的,子进程也修改了

对象方法

方法 说明
current_process 查看进程号
os.getpid() 查看进程号
os.getppid() 查看父进程进程号
terminate() 杀死子进程
is_alive() 判断进程是否存活

terminate()、is_alive()方法结合看不出结果,因为操作系统需要反应时间,主进程睡一下(time.sleep)即可;

terminate()、is_alive()方法结合看不出结果,因为操作系统需要反应时间,主进程睡一下(time.sleep)即可;

————————

Concurrent programming theory

parallel

Parallel means “parallel” or “simultaneous implementation or implementation”. In the operating system, several program segments run in the system at the same time, and the execution of these programs overlaps in time. The execution of one program segment has not ended, and the execution of the other program segment has begun. The programs are executed at the same time, both micro and macro;

Parallel means “parallel” or “simultaneous implementation or implementation”. In the operating system, several program segments run in the system at the same time, and the execution of these programs overlaps in time. The execution of one program segment has not ended, and the execution of the other program segment has begun. The programs are executed at the same time, both micro and macro;

问:单核CPU能否实现并行?
答:不能,但是可以实现并发

Concurrent

Concurrency refers to: in the same time period, two or more programs are executed with time overlap (macro is simultaneous, micro is still sequential execution);
Generally speaking, as long as multiple programs look like running at the same time;

Concurrency refers to: in the same time period, two or more programs are executed with time overlap (macro is simultaneous, micro is still sequential execution);

Generally speaking, as long as multiple programs look like running at the same time;

问:12306可以同一时间支持几个亿的用户买票 问是并行还是并发
答:高并发

The above is based on a single core CPU~

The above is based on a single core CPU~

Procedures and processes

Baidu Encyclopedia: a process is a running activity of a computer program on a data set. It is the basic unit for resource allocation and scheduling of the system and the basis of the structure of the operating system. In the early computer structure of process oriented design, process is the basic executive entity of program; In the contemporary computer architecture of thread oriented design, process is the container of thread. Program is the description of instructions, data and their organizational form, and process is the entity of program.
From a development perspective, a program can be understood as a pile of code, and how does a process come into being?
The generation of process can be understood as: the running program is alive, and the program (a pile of code) is dead there. If it does not run, it will not produce anything;

Baidu Encyclopedia: a process is a running activity of a computer program on a data set. It is the basic unit for resource allocation and scheduling of the system and the basis of the structure of the operating system. In the early computer structure of process oriented design, process is the basic executive entity of program; In the contemporary computer architecture of thread oriented design, process is the container of thread. Program is the description of instructions, data and their organizational form, and process is the entity of program.

From a development perspective, a program can be understood as a pile of code, and how does a process come into being?

The generation of process can be understood as: the running program is alive, and the program (a pile of code) is dead there. If it does not run, it will not produce anything;

Process scheduling in the case of single core

Process scheduling in the case of single core

1、FCFS:先来先服务,如果先来一个长作业,比如要执行24h的程序,后面只有1s的程序,这样对短作业是不友好的;
2、短作业优先调度算法:相反这样是对长作业不友好;
		    |
    		|
             
3、时间片轮转法+多级反馈队列:先分配给新的多个进程相同的时间片段,之后根据进程消耗时间片多少分类执行;

Process three state diagram

Ready, running, blocking

Ready, running, blocking

Concurrent programming

  • Ready state: when the process has allocated all necessary resources except CPU, it can be executed immediately as long as the processor is obtained. The process state at this time is called ready state.
  • Running state: when the process has obtained the processor and its program is executing on the processor, the process state at this time is called running state.
  • Blocking state: when the executing process is unable to execute due to waiting for an event, it abandons the processor and is in a blocking state. There can be many kinds of events that cause process blocking, such as waiting for I / O to complete, the application buffer cannot be satisfied, waiting for a letter (signal), etc.

The program enters the ready state after running until the first line of code runs. If there is a code in the middle that causes the program to tamp, it will lead to the blocking state; The three states will switch continuously;

The program enters the ready state after running until the first line of code runs. If there is a code in the middle that causes the program to tamp, it will lead to the blocking state; The three states will switch continuously;

Synchronous, asynchronous

< strong > synchronization: wait for the return result of the task after submitting the task, and do nothing during the period (consume resources) < / strong >

< strong > asynchronous: after submitting a task, do not wait for the return result of the task, but directly do other things. The result is automatically reminded (optimized) by the feedback mechanism < / strong >

# 概念剖析
比如你要做三件事:学习、洗衣服、做饭
# 同步
同步做这三件事的过程:学习期间不能干别的事,不学习了才能去洗衣服,洗衣服期间只能等着洗衣机洗完衣服,不能干别的事,然后衣服洗好了,才能去做饭;
# 结果就是这样又耗时耗力

# 异步
异步做着三件事的过程:衣服丢洗衣机里,米放锅里,然后去学习;这样是不是很轻松,也省时间;

Blocking, non blocking

Both are used to describe the execution status of a task

Both are used to describe the execution status of a task

< strong > blocking: it means that the current thread will be suspended before the call result is returned < / strong >

比如python中的input方法获取输入,你不输入会一直处于等待状态,也就是阻塞

< strong > non blocking: the opposite of blocking. Execute and run < / strong >

Synchronous / asynchronous, blocking / non blocking combination

  • Synchronous blocking: this form has the lowest efficiency; For example, if you wait in line for dinner, you can only wait and do nothing;
  • Asynchronous blocking: for example, if you wait in line for dinner, just ask the clerk to shout you. During this period, you can buy a cup of milk tea nearby (asynchronous), but you can’t leave the restaurant accessories (blocking);
  • Synchronous non blocking: for example, when you queue up for dinner, you estimate that the queuing time is very long. You open the king glory. You have to move forward while playing games and pay attention to the queuing situation. In this way, the king glory is equivalent to a program. Queuing is a program, and the two need to be switched constantly;
  • Asynchronous non blocking: the efficiency is very high. For example, when you queue up for dinner, you can give the order number to your girlfriend and let her queue up. You go to the toilet (asynchronous) and shit (non blocking); Does this mean that there is no need to queue up or wait next to the restaurant? This is asynchronous and non blocking;

Maybe chestnuts are inappropriate. Don’t tangle

Maybe chestnuts are inappropriate. Don’t tangle

Concurrent programming practice

Create process

Setting up a process in windows is similar to an import module, which executes the code from top to bottom. It must execute the code of setting up the process in the judgment statement;
In Linux, a complete copy of the code is directly executed, and it does not need to be executed in the judgment statement

Setting up a process in windows is similar to an import module, which executes the code from top to bottom. It must execute the code of setting up the process in the judgment statement;

__main__

In Linux, a complete copy of the code is directly executed, and it does not need to be executed in the judgment statement

__main__
# 代码层面创建进程
方法一:
from multiprocessing import Process
import time
import os


def test_print(name):
    print(os.getpid())  # 获取进程号
    print(os.getppid())  # 获取父进程号
    print('%s正在运行' % name)
    time.sleep(3)
    print('%s已经结束' % name)


if __name__ == '__main__':
    p = Process(target=test_print, args=('Hammer',))  # 生成一个进程对象
    p.start()  # 告诉操作系统开设一个新的进程,异步提交
    print(os.getpid())
    print('主进程')
    
    
方法二: 采用继承的方法
class MyProcess(Process):
    def __init__(self, name):
        super().__init__()
        self.name = name

    def run(self):
        print('%s正在运行' % self.name)
        time.sleep(3)
        print('%s已经结束' % self.name)

if __name__ == '__main__':
    p = MyProcess('Hammerze')
    p.start()
    print('主进程')

Join method of process

First, execute the code of the child process, and then execute the code of the parent process after the child process is executed;
Equivalent to changing to serial;

First, execute the code of the child process, and then execute the code of the parent process after the child process is executed;

Equivalent to changing to serial;

from multiprocessing import Process
import time


def test(name, n):
    print('%s is run' % name)
    time.sleep(n)
    print('%s is over' % name)


if __name__ == '__main__':
    p_list = []
    start_time = time.time()
    for i in range(1, 4):
        p = Process(target=test, args=(i, i))
        p.start()
        p_list.append(p)
        p.join()  # 串行  9s+
   #  for p in p_list:     
   #      p.join()
   #  print(time.time() - start_time)

No interaction between processes by default

# 进程间数据是相互隔离的
from multiprocessing import Process

money = 100

def test():
    global money
    money = 999


if __name__ == '__main__':
    p = Process(target=test)
    p.start()
    # 先确保子进程运行完毕了 再打印
    p.join()
    print(money)  # 100,打印的是主进程的,子进程也修改了

Object method

方法 说明
current_process 查看进程号
os.getpid() 查看进程号
os.getppid() 查看父进程进程号
terminate() 杀死子进程
is_alive() 判断进程是否存活

terminate()、is_ The combination of the alive () method can’t see the result, because the operating system needs reaction time, and the main process can sleep for a while (time. Sleep);

terminate()、is_ The combination of the alive () method can’t see the result, because the operating system needs reaction time, and the main process can sleep for a while (time. Sleep);