Team:XMU-China/Software
Our team has developed an app, which makes a combination of AI algorithm and blockchain. It is designed in order to better control the hardware, making it more convenient and handy for grass-root hospitals and their patients, and facilitate the doctors' diagnosis. More details about the principle of the software, please switch to our code on the Github.
Figure1. App interface
1. The Hardware Control
The first model mainly targets for the specific operations of the hardware.
(1) Engine Start Stop:
Users could click the button, “Engine Start Stop”, to automatically start the hardware, and complete a series of tasks including centrifugation and pictures-taking.
(2) Step – Control:
a. Two-Sliders Controller: Being used to set the time and speed of the hardware engines.
b. The On-off Controller: Being used to start the hardware and send the speed and time the hardware needed for operating.
c. The Pictures-taking Controller: used to control the pick-up head of the camera to take photos.
(3) Show the real-time speed of the engine.
2. The detection of the images
The second model is mainly responsible for the analysis of pictures taken by the camera.
(1) The Loading Controller:
Showing the pictures taken after the operation of the hardware.
(2) Analysis Controller:
Obtain the aperture gaps of the fluorescence and corresponding diseases’ information through analyzing those taken pictures.
3. Diagnosis
The third model mainly serves to deploy the cases of patients, let doctors check those cases and give diagnoses in time.
(1) Setting an Account:
Establish blockchain accounts, and input the names、keywords and identities. After that, users can obtain the addresses of their accounts on Ethereum.
(2) Buying currencies and check the balance:
Which is used for the consumption of deploying cases for miners.
(3) Deploying cases:
Fill in related information according to all the intelligent-treaty-templets of cases, so that the information in cases could be published on Ethereum in the form of intelligent treaty.
(4) Check the Cases:
Doctor users can check all the cases deployed by patients, and make diagnoses accordingly.
Based on the improved C/S(Client/Server) structure, we set up a cloud server for exchanging information between software and hardware, storing database information and realizing related functions. With the rapid development of the Internet, mobile office and distributed office are becoming more and more popular.
Considering that the scale of our hardware will involve grassroots hospitals in towns and villages, it is a necessity for our system to be more scalable. In order to avoid the limitations of being applicable to LAN only, and to enhance the universality of software applications, we put the server in the cloud. Through the cloud LAN, terminals from LANs in different networks can be connected to form a new LAN, so that all internal connections are made.
The terminal devices can access on another, which enables remote control of the hardware by our software. Different from the traditional C/S structure, we put the data in the cloud server. The business logic of the software is also implemented only by the server.
The app client does not save any business data and database connection information, which ensures the security to some extent, and also ensures the real-time and consistency of the data. In the future, if the number of users increases gradually, we can increase the number of servers by establishing a cluster server system smoothly. During this process, what we need to do is only realizing the load balance between each server, and it is unnecessary to give up the original server. The App is written with the react native framework, which supports android and ios, and has strong compatibility.
Figure2. The communication structure
Hardware control communication: Both the app part and the hardware part communicate with the server by using the web-socket protocol. The control messages are sent to the hardware from the app end, and the hardware realizes the function after being transmitted to the hardware module by the server. After that, the corresponding results are returned to the server. And a series of analyses and operations are carried out on the server side before finally transmitting back to the app.
Image recognition algorithm: We used Convolutional neural network CNN algorithm in deep learning algorithm. Through the picture taken by the hardware, we need to analyze which holes fluoresce in the picture to determine the corresponding diseases.
Figure3. Pictures' transfer and conversion process
Firstly, we trained through the data sets which we had collected and got the model in advance. Then, we used the model to classify the photo taken by the hardware to obtain the specific pathological types to which the photo belonged. Because there were too many parameters in the fully connected layer of the fully connected neural network, not only will the calculation speed slow down, but also the over-fitting problem will occur. Therefore, we used the convolutional neural network structure to classify the image. The convolutional neural network architecture which we used is shown in the following picture:
Figure4. The convolutional neural network architecture
First, the input layer. We transformed the captured image on the server and converted the pixel to 32*32. Since it was the RGB color mode, we inputted a 32*32*3 three-dimensional matrix. Starting from the input layer, the three-dimensional matrix of the previous layer will be transformed into the next-level three-dimensional matrix through the neural network structure, and will finally reach connection layer.
After that, we performed a four-layer convolution operation. Through the convolution layer processing, the depth of the node matrix will increase. When it came to the corresponding pooling layer, the size of the matrix would be reduced, which further reduced the number of nodes in the last fully connected layer, thereby reducing the parameters in the entire nerve network.
Then, the features of the image were extracted through two fully connected layers, and the final classification result was given. Finally, the probability distributions of the pictures’ belonging to the different examples was obtained through the softmax layer.
Convolution layer. In the convolutional layer, we used a filter to convert a sub-node matrix of the current layer neural network into a unit node matrix of the next layer of neural networks. This simulated the operating process of a convolutional layer:
Figure5. The operating process of a convolutional layer
$g(i)=f(\sum_{x=1}^{X}\sum_{y=1}^{Y}\sum_{z=1}^{Z}a_{x,y,z}\times w^{i}_{x,y,z}+b^i)$
The value of the i-th node in the identity matrix is $g(i), w^{i}_{x, y, z}$，which represents the filter input node for the i-th node in the output unit node matrix (x, y, z). b^i represents the offset term parameter corresponding to the i-th output node. And f is the activation function. In this project we choose 5*5 and 3*3 filter size and used ReLU as the activation function.
Pool layer. The pooling layer can effectively reduce the size of the matrix, thereby reducing the parameters in the final fully connected layer. Using a pooled layer can both speed up calculations and prevent overfitting problems. The pooling layer that uses the maximum operation is called the max pooling layer. In this project we used the max pooling layer.
Figure6. The operating process of a pooling layer
Through training, we can see that the loss value of the model is convergent, the accuracy rate can reach 99.90%, and the model is reliable.
Figure7. Model's training process
Figure8. Loss function curve
Why did we choose block chain technology?
The block chain accelerates the arrival of the“digital credit society” with its characteristics of decentral, reliable, immutable, trustworthy and anonymous. In the medical industry, what we need more is a highly trusted mechanism to protect the rights of the patients. Therefore, we chose block chain technology which enabled the patients to deploy their own medical records, and according to which doctors carried out the diagnoses.
Based on the block chain technology under Ethereum, the pow consensus mechanism was used. When generating a block, the system allows all the nodes to calculate a random number fairly. The node of the random number which was found first will be the producer of the block, and will get the corresponding block reward. Since the hash function is a hash function, the only way to solve a random number mathematically is through the method of exhaustion. The randomness is very good, and everyone can participate in the execution of the protocol.
Due to the setting of the Merkel tree’s root, the verification process of the solution of the hash function can also be implemented quickly. Therefore, everyone can participate without the permission of a centralized authority, and each participant does not need identity authentication. It can be said that the pow consensus mechanism algorithm is simple and easy to implement. The nodes can also enter freely and decentralize to a high extent. Destroying the system requires a huge investment. Therefore, the system could be said as extremely safe. The seletion of the block producer is realized by solving the hash function by the node. The final process of generating and verifying the proposal to the consensus is purely a mathematical problem, and the nodes can reach a consensus without exchanging additional information.
We set up an Ethereum private chain belonging to our system on the server. Every doctor and patient acts as an account on the block chain node. Everyone will get an address on the private chain of Ethereum through the passwords set by themselves. The process of generating this address is generating a private key from the secp256k1 curve first, which is composed of random 256 bits, and then use the elliptic curve digital signature algorithm (ECDSA) to map the private key into a public key. After that, the public key becomes 256bit through the Keccak-256 one-way hash function, and then takes 160 bits as the address.
Smart contracts can be deployed to private chains through addresses. The smart contract is a computer protocol designed to disseminate, verify or execute contracts in an informational way. In our system, we wrote the medical record in the form of a smart contract as a template, and the users can fill in personal information and medical record information. Therefore, the doctors can view all the medical records deployed to the private chain. That is to say, the doctors and can even choose from the medical records to make diagnoses, and write their diagnosing results into the medical records deployed by the patients. In intelligence contracts, we have written a one-on-one mechanism for doctors and patients, that is, one patient can only have one doctor to receive the consultation. Once the patient is admitted, other doctors will not be able to modify the diagnoses.
In block chain technology, the hash algorithm is mainly used to receive a plaintext and convert it into an output hash with a short length and a fixed number of bits in an irreversible way. Because this encryption process is irreversible, it means that no information related to the original text can be inferred by outputting the contents of the hash. Any change in the input information, even a single change of the digit, will result in a significant change in the hash result. Based on the one-to-one correspondence between the output hash and the input text, the hash algorithm can be used to verify whether the information has been modified.
Due to fact that the block chain technology could not be modified, once the case is viewed, the authenticity and accuracy of the case information can be guaranteed. Once there is a doctor-patient conflict or a medical consultation, we can refer to the accountability system, and therefore protect the rights of both the doctors and the patients. The information of each user and case address, together with related information is stored in the server's levedb database. In the future, we can also access the public health system database to realize the popularization.
Figure9. Interactive process
