发表于 | 更新于 | 分类于

保存和恢复模型, 分为两个部分: 1.模型的结构(即创建模型的代码), 2.模型的训练权重(即参数), 所以, 不能总只是记着参数而没有结构. 另, 保存TF代码中的模型有很多方法(取决于API), 本文中使用的是tf.keras, 是TF中的”高阶API”. (这会区别于TF中”原始”的保存和加载代码) P.S. 看官方的东西, 踏实的有点过分.

阅读全文 »

发表于 | 更新于

发表于 | 更新于 | 分类于

训练首个神经网络: 基本分类(个人总结)

项目背景: 数据集是FashionMNIST(70000张28*28灰度图像(训练6/测试1), 含衣服着装的10个类别), 此数据集本身已经被包含在keras中, 故只要导入keras.datasets.fashion_mnist模块, 调用函数fashion_mnist.load_data(), 便可以得到: 训练集数据/标签, 测试集数据/标签的变量(图片由视觉平面变成了np.array()的编程数据格式).

阅读全文 »

发表于 | 更新于 | 分类于

MATPLOTLIB_乐园

REF1) => REF2 => REF3

plt.figsize: https://www.baidu.com/s?ie=UTF-8&wd=plt.figsize => https://blog.csdn.net/weixin_43685844/article/details/88982811

pyplot画板思路: 先准备好数据()

1
2
3
4
5
6
graph TD
    id[带文本的矩形]
    id4(带文本的圆角矩形)
    id3>带文本的不对称的矩形]
    id1{带文本的菱形}
    id2((带文本的圆形))
1
2
3
graph LR
   A[准备好数据] --> B["将数据喂给figure画板(此时还没有真的绘制)"]
   B-->C["plt.show()才是将绘制方案显示在屏幕上的一步"]

[]

发表于 | 更新于 | 分类于

NUMPY_LIB_PLAYGROUND

切片 - None, …(三个点), 与多维度

很是疑惑, 便记下(参考[REF]).

首先, None关键字出现在列表的切片中, 表示所在维度位置上新增一个维度(None的一个别名是numpy.newaxis), 验证如下:

1
2
3
4
>>> import numpy as np
>>> np.newaxis			# 此处因为是None, 所以不会有结果显示的;
>>> type(np.newaxis)
<class 'NoneType'>

验证None在列表切片上新增维度的作用:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
>>> a = np.arange(25).reshape((5,5))
>>> a
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24]])
>>> print(a[:, None],'\n\nShape is: ', a[:, None].shape)
[[[ 0  1  2  3  4]]

 [[ 5  6  7  8  9]]

 [[10 11 12 13 14]]

 [[15 16 17 18 19]]

 [[20 21 22 23 24]]] 

Shape is:  (5, 1, 5)

上面的代码中, None作用在第二维上(a[:, None]), 所以PYTHON直接在第二维上增加了一个维度: 原先的第二维是0, 1, 2, 3, 4共5个元素, 而None增加维度之后, 第二维变成了[ 0 1 2 3 4], 只有1个列表元素, None把原先第二维的5个元素升了一个维度(加了[]), 而并没改变原先的5个元素, 所以数组a的维度是这么增加了, 故, 如是记: None放在哪一维, 就会在哪一维上出现新的维度. 再看:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
>>> a[:, :, None]
array([[[ 0],
        [ 1],
        [ 2],
        [ 3],
        [ 4]],

       [[ 5],
        [ 6],
        [ 7],
        [ 8],
        [ 9]],

       [[10],
        [11],
        [12],
        [13],
        [14]],

       [[15],
        [16],
        [17],
        [18],
        [19]],

       [[20],
        [21],
        [22],
        [23],
        [24]]])
>>> a[:, :, None, None]		# :(维度1), :(维度2), None(维度3), None(维度4);
array([[[[ 0]],

        [[ 1]],

        [[ 2]],

        [[ 3]],

        [[ 4]]],


       [[[ 5]],

        [[ 6]],

        [[ 7]],

        [[ 8]],

        [[ 9]]],


       [[[10]],

        [[11]],

        [[12]],

        [[13]],

        [[14]]],


       [[[15]],

        [[16]],

        [[17]],

        [[18]],

        [[19]]],


       [[[20]],

        [[21]],

        [[22]],

        [[23]],

        [[24]]]])

好了, 不能再None下去了, 这排版已经不能看了, 我们试试没那么难看但会出错的例子:

1
2
3
4
>>> a[:, :, None, :]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: too many indices for array

上面的代码, 出错了, 因为在第四维我们用了:来索引所有元素, 而array a并没有具体的第四维(None是维度扩充, 所以有没有所在位置的维度, 对None来说, 不会造成语法和逻辑上的错误).

而切片中. . .(三个点, 此处点之间含空格是为了排版)是什么意思? 它表示省略所有冒号的意思了:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
>>> a[..., None]
array([[[ 0],
        [ 1],
        [ 2],
        [ 3],
        [ 4]],

       [[ 5],
        [ 6],
        [ 7],
        [ 8],
        [ 9]],

       [[10],
        [11],
        [12],
        [13],
        [14]],

       [[15],
        [16],
        [17],
        [18],
        [19]],

       [[20],
        [21],
        [22],
        [23],
        [24]]])
>>> a[..., None].shape
(5, 5, 1)
>>> a[None, ...]
array([[[ 0,  1,  2,  3,  4],
        [ 5,  6,  7,  8,  9],
        [10, 11, 12, 13, 14],
        [15, 16, 17, 18, 19],
        [20, 21, 22, 23, 24]]])
>>> a[None, ...].shape
(1, 5, 5)

打住打住, 一切皆空, 一切皆None(包括前面那一句话).

Axis - 轴

关于Numpy中的axis(轴), 还是看官方文档的示例吧(“The N-dim array, Calculation节”[REF]), 里面讲的例子, 够简单概括和准确(相对于博文来说):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# Example of the axis argument
# A 3-dimensional array of size 3 x 3 x 3, summed over each of its three axes
>>> x
array([[[ 0,  1,  2],
        [ 3,  4,  5],
        [ 6,  7,  8]],
       [[ 9, 10, 11],
        [12, 13, 14],
        [15, 16, 17]],
       [[18, 19, 20],
        [21, 22, 23],
        [24, 25, 26]]])
>>> x.sum(axis=0)
array([[27, 30, 33],
       [36, 39, 42],
       [45, 48, 51]])
>>> # for sum, axis is the first keyword, so we may omit it,
>>> # specifying only its value
>>> x.sum(0), x.sum(1), x.sum(2)
(array([[27, 30, 33],
        [36, 39, 42],
        [45, 48, 51]]),
 array([[ 9, 12, 15],
        [36, 39, 42],
        [63, 66, 69]]),
 array([[ 3, 12, 21],
        [30, 39, 48],
        [57, 66, 75]]))

以上示例中, 想要弄清楚axis=0/1/2分别有什么结果, 先把array的shape找到来, 上例中, 为: (3, 3, 3). 其中:

  • 第一个3与axis=0相关联, 表明array([e1,e2,e3])中最外围[]内元素的轴向, axis=0是作用于这个维度上的e1,e2,e3的, 有英文across之意;
  • 第二个3与axis=1相关联, 表明第二个shape维度中元素的轴向, 即array([[e1,e2,e3],[e1,e2,e3],[e1,e2,e3]]), axis=1是作用于这个维度上的e1,e2,e3的;
  • 第三个3与axis=2相关联, 表明第三个shape维度中元素的轴向, 即array([[[e1,e2,e3],[...],[...]],[[e1,e2,e3],[...],[...]],[[e1,e2,e3],[...],[...]]]), axis=2是作用于这个维度上的e1,e2,e3的.

有了以上的解释, 上述官方代码中(x.sum(0), x.sum(1), x.sum(2))的求和结果就不难解释了: 它们对指定的axis上被across的所有元素求和. (注: 轴是个抽象概念, 不直观, 不直接, 只要能弄清逻辑就行, 慢一点正常的, 换句话说: 能很快被理清逻辑的, 那还能算抽象吗)

NumPy User Guide[REF]

NumPy basics[REF]

Data types[REF]

Indexing[REF]

Others - 其余小叙

numpy.load() 函数[REF]

函数 说明
numpy.load(file, mmap_mode=None, allow_pickle=False, fix_imports=True, encoding='ASCII') Load arrays or pickled objects from .npy, .npz or pickled files

load()返回的是: array, tuple, dict, etc, 即”data stored in the file”. (For .npz files, the returned instance of NpzFile class must be closed to avoid leaking file descriptors) 以下是一些例子:

1
2
3
4
5
# Store data to disk, and load it again;
>>> np.save('/tmp/123', np.array([[1, 2, 3], [4, 5, 6]]))
>>> np.load('/tmp/123.npy')
array([[1, 2, 3],
       [4, 5, 6]])
1
2
3
4
5
6
7
8
9
10
11
# Store compressed data to disk, and load it again;
>>> a=np.array([[1, 2, 3], [4, 5, 6]])
>>> b=np.array([1, 2])
>>> np.savez('/tmp/123.npz', a=a, b=b)
>>> data = np.load('/tmp/123.npz')
>>> data['a']			# 看此处的字典key值, 应为savez()的参数"a=a"中前面那个'a';
array([[1, 2, 3],
       [4, 5, 6]])
>>> data['b']			# 看此处的字典key值, 应为savez()的参数"b=b"中前面那个'b';
array([1, 2])
>>> data.close()
1
2
3
4
# Mem-map the stored array, and then access the second row directly from disk;
>>> X = np.load('/tmp/123.npy', mmap_mode='r')
>>> X[1, :]				# Second row;
memmap([4, 5, 6])

号外: PYTHON中的Special Methods

For standard library functions:

ndarray.__copy__() Used if copy.copy is called on an array.
ndarray.__deepcopy__() Used if copy.deepcopy is called on an array.
ndarray.__reduce__() For pickling.
ndarray.__setstate__(state, /) For unpickling.

Basic customization:

ndarray.__new__(*args, **kwargs) Create and return a new object.
ndarray.__array__() Returns either a new reference to self if dtype is not given or a new array of provided data type if dtype is different from the current dtype of the array.
ndarray.__array_wrap__()

Container customization: (see Indexing)

ndarray.__len__(self, /) Return len(self).
ndarray.__getitem__(self, key, /) Return self[key].
ndarray.__setitem__(self, key, value, /) Set self[key] to value.
ndarray.__contains__(self, key, /) Return key in self.

Conversion; the operations int, float and complex. . They work only on arrays that have one element in them and return the appropriate scalar.

ndarray.__int__(self)
ndarray.__float__(self)
ndarray.__complex__()

String representations:

ndarray.__str__(self, /) Return str(self).
ndarray.__repr__(self, /) Return repr(self).

numpy.squeeze() 函数[REF]

函数 说明
numpy.squeeze(a, axis=None) 将输入数组a中的长度为1的维度(single-dimensional entries)移除掉.

返回: ndarray(squeezed), 即: the input array, but with all or a subset of the dimensions of length 1 removed. This is always a itself or a view into a.

抛出(Raises): ValueError, If axis is not None, and an axis being squeezed is not of length 1.

具体例子: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
>>> x = np.array([[[0], [1], [2]]])
>>> x.shape
(1, 3, 1)

>>> np.squeeze(x)
array([0, 1, 2])
>>> np.squeeze(x).shape
(3,)

>>> np.squeeze(x, axis=0)
array([[0],
       [1],
       [2]])
>>> np.squeeze(x, axis=0).shape
(3, 1)

>>> np.squeeze(x, axis=1).shape
Traceback (most recent call last):
...
ValueError: cannot select an axis to squeeze out which has size not equal to one

>>> np.squeeze(x, axis=2)
array([[0, 1, 2]])
>>> np.squeeze(x, axis=2).shape
(1, 3)

numpy.reshape() 函数[REF]

函数 说明
numpy.reshape(a, newshape, order=’C’) Gives a new shape to an array without changing its data.

返回 : ndarray(reshaped_array), this will be a new view object if possible; otherwise, it will be a copy. (Note: there is no guarantee of the memory layout (C- or Fortran- contiguous) of the returned array)

(See also: ndarray.reshape, Equivalent method)

1
2
3
4
5
6
7
8
9
10
# 例子:
>>> a = np.array([[1,2,3], [4,5,6]])
>>> np.reshape(a, 6)						# 默认的order是'C': C-like index ordering;
array([1, 2, 3, 4, 5, 6])
>>> np.reshape(a, 6, order='F')	# Fortran-like index ordering;
array([1, 4, 2, 5, 3, 6])
>>> np.reshape(a, (3,-1))       # The unspecified value is inferred to be 2;
array([[1, 2],
       [3, 4],
       [5, 6]])

numpy.random.randn() 函数[REF]

函数 说明
numpy.random.randn(d0, d1, …, dn) Return a sample (or samples) from the “standard normal” distribution.

参数: d0, d1, …, dn(int, optional), the dimensions of the returned array, should be all positive. If no argument is given a single Python float is returned. (注: 返回结果dtype是float, 从”a univariate “normal” (Gaussian) distribution of mean 0 and variance 1”中采样得来)

返回: ndarray or float(Z), a (d0, d1, …, dn)-shaped array of floating-point samples from the standard normal distribution, or a single such float if no parameters were supplied.

(注: For random samples fromN(\mu, \sigma^2), use: sigma * np.random.randn(...) + mu)

(注: See also standard_normal, similar, but takes a tuple as its argument)

具体例子:

1
2
3
4
5
6
>>> np.random.randn()
2.1923875335537315	# random;
# Two-by-four array of samples from N(3, 6.25);
>>> 2.5 * np.random.randn(2, 4) + 3
array([[-4.49401501,  4.00950034, -1.81814867,  7.29718677],  #random
       [ 0.39924804,  4.68456316,  4.99394529,  4.84057254]]) #random

numpy.clip() 函数[REF]

函数 说明
numpy.clip(a, a_min, a_max, out=None, kwargs) Clip (limit) the values in an array. Given an interval, values outside the interval are clipped to the interval edges. For example, if an interval of [0, 1] is specified, values smaller than 0 become 0, and values larger than 1 become 1.

返回: ndarray(clipped_array), an array with the elements of a, but where values < a_min are replaced with a_min, and those > a_max with a_max.
(注: out参数是ndarray, 可选, 函数的结果会placed在这个array中, “it may be the input array for in-place clipping”)

具体例子:

1
2
3
4
5
6
7
8
9
10
11
12
>>> a = np.arange(10)
>>> np.clip(a, 1, 8)
array([1, 1, 2, 3, 4, 5, 6, 7, 8, 8])
>>> a
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> np.clip(a, 3, 6, out=a)
array([3, 3, 3, 3, 4, 5, 6, 6, 6, 6])
>>> a = np.arange(10)
>>> a
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> np.clip(a, [3, 4, 1, 1, 1, 4, 4, 4, 4, 4], 8)		# a_min是一个数组;
array([3, 4, 2, 3, 4, 5, 6, 7, 8, 8])

TODO: Array creation routines

太累了我不想写: [REF].

PEP与PYHTON之禅(ZEN OF PYTHON)

PEP是Python Enhancement Proposal的缩写, 是”PYTHON增强提案”的意思, PEP的背景是: Python核心开发者通过邮件列表讨论有关于Python各种方面的问题/提议/计划, 在这些交流中, 这些问题/提议/计划经过了核心开发者审阅(review)和认可后, 最终形成的正式文档(即PEP), 起到了对外公示的作用.

PEP的官方地址是[REF], 也是PEP0的地址, 这个PEP其实是所有PEP的索引(“Index of Python Enhancement Proposals (PEPs)”), 其中PEP20[REF], 便是著名的”PYTHON之禅“. 如, 以下的增强提案, 其涉及的内容都会包含在我们使用的Python代码中(#渔#与#所以然#):

PEP Index 说明
PEP 318 Decorators for Functions and Methods: 关于装饰器.
PEP 282 A Logging System: 关于Logging标准库.
PEP 3101 Advanced String Formatting: 字符串格式化.
PEP 380 Syntax for Delegating to a Subgenerator: 引入“yield from”语法.
PEP 492 Coroutines with async and await syntax ：引入async/await语法.
PEP 3135 New Super ：Python3中的super用法.

Ref: [博文1], [博文2], 以及[10个必读].

numpy.append() 函数[REF]

函数 说明
numpy.append(arr, values, axis=None) 将一个array附加到另一个array中(append values to the end of an array).
参数 参数说明
arr : array_like Values are appended to a copy of this array.
values : array_like These values are appended to a copy of arr. It must be of the correct shape (the same shape as arr, excluding axis). If axis is not specified, values can be any shape and will be flattened before use.
axis : int, optional The axis along which values are appended. If axis is not given, both arr and values are flattened before use.
返回: append : ndarray A copy of arr with values appended to axis. Note that append does not occur in-place: a new array is allocated and filled. If axis is None, out is a flattened array. 
1
2
3
4
5
6
7
8
9
10
11
12
# 具体例子;
>>> np.append([1, 2, 3], [[4, 5, 6], [7, 8, 9]])
array([1, 2, 3, 4, 5, 6, 7, 8, 9])		# Axis is None, flattened;
# When axis is specified, values must have the correct shape;
>>> np.append([[1, 2, 3], [4, 5, 6]], [[7, 8, 9]], axis=0)	# 两个array的shape相同;
array([[1, 2, 3],
       [4, 5, 6],
       [7, 8, 9]])
>>> np.append([[1, 2, 3], [4, 5, 6]], [7, 8, 9], axis=0)		# 两个array的shape不同;
Traceback (most recent call last):
...
ValueError: arrays must have same number of dimensions

numpy.resize() 函数[REF]

函数 说明
numpy.resize(a, new_shape)[source] Return a new array with the specified shape. (关键在于PYTHON是怎么填充/重排数组内的元素的)
Parameters Explaination
a: array_like Array to be resized.
new_shape: int or tuple of int Shape of resized array.
Returns: reshaped_array: ndarray The new array is formed from the data in the old array, repeated if necessary to fill out the required number of elements. The data are repeated in the order that they are stored in memory. 
1
2
3
4
5
6
7
8
9
10
# 具体例子;
>>> a=np.array([[0,1],[2,3]])
>>> np.resize(a,(2,3))
array([[0, 1, 2],				# Data repeated;
       [3, 0, 1]])
>>> np.resize(a,(1,4))
array([[0, 1, 2, 3]])
>>> np.resize(a,(2,4))	# Data repeated;
array([[0, 1, 2, 3],
       [0, 1, 2, 3]])

numpy.trim_zeros() 函数[REF]

函数 说明
numpy.trim_zeros(filt, trim=’fb’) Trim the leading and/or trailing zeros from a 1-D array or sequence.
Parameters Explaination
filt: 1D array or squence Input array.
trim: str, optional A string with ‘f’ representing trim from front and ‘b’ to trim from back. Default is ‘fb’, trim zeros from both front and back of the array.
Returns: trimmed: 1D array or squence The result of trimming the input. The input data type is preserved. 
1
2
3
4
5
6
7
8
9
# 具体例子;
>>> a = np.array((0, 0, 0, 1, 2, 3, 0, 2, 1, 0))
>>> np.trim_zeros(a)
array([1, 2, 3, 0, 2, 1])
>>> np.trim_zeros(a, 'b')
array([0, 0, 0, 1, 2, 3, 0, 2, 1])
# The input data type is preserved, list/tuple in means list/tuple out;
>>> np.trim_zeros([0, 1, 2, 0])
[1, 2]

发表于 | 更新于 | 分类于

TF_APIs_Used_In_Code

tf.nn.conv2d[ref]: 这是最先去查找的API函数(足见其使用程度), 它定义在 tensorflow/python/ops/nn_ops.py中, 它对所给的4D input和filter张量(Tensor)进行2D的卷积运算(这里的2-D和4-D别搞混了, 一个是卷积, 一个是Tensor的尺寸), 它有别名tf.compat.v1.nn.conv2d和tf.nn.conv2d, 它的函数形式为:

阅读全文 »

发表于 | 更新于 | 分类于

0.简介

此札记的目的还是, 通过输出: 1.理解概念, 2.加深印象, 3.融会贯通.

首先, 先看一段Python中Tensorflow的代码:

阅读全文 »

发表于 | 更新于 | 分类于

AlexNet的”官方”论文为ImageNet Classification with Deep Convolutional Neural Networks [ref], 发表于2012(7年前, :)), 论文的三位作者分别是Alex(第一作者), Ilya和Geoffrey Hinton(前两位的导师), 其中的Dropout概念, 应当是很有影响的概念了.

阅读全文 »

发表于 | 更新于 | 分类于

　　现有最基本最基本的Python概念点(看一遍都懂, 就记不住)整理了, Mark在文章中, 日后有更丰富的经验, 可以以此为基础添加上去, 使枝叶丰茂. (P.s. 重要的是, 摆脱Java和C++等编译语言的长期影响!)

阅读全文 »