Trinity College Dublin

Skip to main content.

Top Level TCD Links

Transformer Load Minimisation in Smart-Grids using Transfer Learning

Controlling a system as complicated as the smart-grid can not be done centrally, as it can consist of hundreds of thousands of devices. Devices must be able to make decisions locally based on the information available to them. Over time they can learn how successive decisions can help them achieve their aims. Each of these devices must meet its own goals (e.g. an Electric Vehicle (EV) must be charged by morning) but it must also satisfy goals of the overall grid. In accomplishing the grid's objective, significant improvements in operational efficiency can be achieved. For example, if the grid tells individual devices that there is a surplus of energy --perhaps because it is a particularly windy day--, they can operate earlier than they otherwise would have, this means the surplus can be used rather than being stored or discarded as would otherwise have been the case.

 In much of our work on energy demand management, we use Multi Agent Systems (MAS).  In this approach each device is controlled by an agent (a decision making entity). By representing each device as an agent, they can form teams and learn better ways to reach their goals than they could on their own.  However, learning in MAS takes a significant amount of time due to the dimensionality of the problem. This work investigates a Transfer Learning based approach with the aim of reducing the learning time. 

EVs place significant load on transformers as they tend to use the same charging pattern, e.g., charging when arriving from work or charging at midnight. Each device has similar goal (e.g., minimize the cost of charging), so there is scope to share knowledge and thereby minimise the learning time. Reductions in learning time lead to increases in performance, in this case, smoother loading of the transformer.

Transfer Learning (TL)

In TL, a process gets information from a source agent, translates it, and transfers it to a target agent. This is done off-line from learning.

Parallel Transfer Learning (PTL)

In PTL, any process can function as a source, target or both at any one time. Agents transfer information at runtime, rather than off-line as in traditional Transfer Learning. Information is transferred only when it is deemed ready to share by the source task.

Fig 1. Transfer learning Fig 2. Parallel transfer learning

PTL Algorithm

    Source Agent

    While (information to share)
    Select target agent from set of neighbours
    Select state-action pair to transfer
    Translate pair via inter-task mapping
    Send to target

Target Agent

While (information to process)
Get local state-action pair
If(not visited) accept neighbour's pair
Else decide whether information is more
accurate
If so accept it and merge it with local information

Experimental Scenario

Six EVs to be controlled by agents. The goal is to keep electrical demand from EVs and uncontrollable loads below a threshold while ensuring EVs are sufficiently charged to meet daily demand.

Evaluation

The graph shows a comparison between three different approaches, using Sequential Transfer Learning; Parallel Transfer Learning and no Transfer Learning.  As would be expected there is an initial performance penalty for using PTL, once this is overcome, it goes on to be the most effective method.  The initial penalty is due to negative transfer while exploring.

Conclusion

Using PTL in MAS has the advantage of removing the requirement for learning a relevant source task prior to executing the target task. It takes advantage of the relatedness of tasks in a MAS. Agents are allowed to cooperatively explore the state-space, reducing the performance penalty for each agent. Rare events experienced by one agent can be very useful to the system as a whole through efficient sharing.

Worse initial performance is sometimes experienced as there is no initial knowledge, the length of time to overcome this is application dependent. Greater amounts of communication and processing are required than in traditional Transfer Learning.

People

Adam Taylor, Ivana Dusparic, Edgar Galván-López, Siobhán Clarke, Vinny Cahill

Sponsors

                                                                                           

 

 


Last updated 14 January 2019 futurecities@tcd.ie (Email).